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Forsword 


Since  1958,  the  MITRE  Corporation  has  fulfilled  a  unique  systems 
engineering  role  for  the  United  States  government.  Our  purpose  is  to 
enhance  the  nation’s  security  and  to  otherwise  further  the  public 
interest  through  scientific  research,  support  for  systems  development 
and  acquisition,  and  technical  advisory  services.  The  scope  of  our 
activities  includes  national  security  matters  in  the  area  of  command, 
control,  communications,  and  intelligence  (C^I),  as  well  as  civil  sys¬ 
tem  areas  for  the  public  benefit. 

As  an  independent,  not-for-profit  corporation,  MITRE  operates 
federally  funded  research  and  development  centers  for  the  Depart¬ 
ment  of  Defense  and  the  Federal  Aviation  Administration.  MITRE 
neither  works  for  nor  competes  with  profit-oriented  industry;  we 
often  serve  as  an  impartial  link  between  our  government  sponsors 
and  competitive  industry.  We  strive  to  bring  together  the  expertise 
and  outlook  of  government,  industry,  and  academia  to  solve  complex 
technical  problems  that  cannot  be  solved  by  any  one  group  alone. 

This  book  shows  the  dimension  of  MITRE’s  services  and  the  scope 
of  our  knowledge  as  applied  to  C^I  programs  for  the  Air  Force.  It 
provides  our  employees  and  sponsors  alike  with  an  historical  perspec¬ 
tive  of  why  the  Corporation  was  formed  and  describes  the  breadth  and 


depth  of  our  role  as  systems  engineer  on  C^I  acquisition  programs.  In 
addition  to  serving  as  an  overview  of  some  of  the  Corporation’s  most 
challenging  work,  the  book  also  contains  practical  advice  that  can  be 
applied  to  future  systems  engineering  initiatives.  The  author — a 
founder  of  the  Corporation  and  a  knowledgeable  systems  engineer — 
played  many  roles,  from  project  leader  for  several  Air  Force-sponsored 
C’l  programs  to  Vice  President. 

Over  the  last  35  years,  MITRE  has  helped  to  develop  many  of  the 
nation’s  CT  systems.  Our  work  program  evolves  continually,  accord¬ 
ing  to  changes  in  national  needs  and  priorities.  Through  our  highly 
competent  staff,  state-of-the-art  facilities,  and  challenging  work 
program,  we  uphold  our  commitment  to  total  quality.  And  while 
technology,  threats,  and  national  priorities  may  change,  MITRE’s 
commitment  to  effective  systems  engineering  remains  firm.  We  will 
continue  to  apply  our  technical  expertise  to  C^I  systems  throughout 
the  post-Cold  War  era  to  the  year  2000  and  beyond. 

Harold  W.  Sorenson 
Bedford  Group  Vice  President 
and  Air  Force  FFRDC  Director 
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Introduction 

Each  day,  sophisticated  information  systems  provide  the  U.S.  with 
crucial  capabilities  both  to  understand  the  world  situation  and  to 
react  effectively  as  required  by  our  nation’s  decision  makers.  These 
systems  attest  to  the  success  of  the  cooperative  efforts  of  government 
and  industry.  Over  the  last  35  years,  to  help  provide  those  capabili¬ 
ties,  The  MITRE  Corporation  has  been  privileged  to  fulfill  the  role  of 
systems  engineer  on  more  than  100  different  command,  control, 
cOininua;«.arions,  and  iiitc;iij,ence  (C^I)  >ys:rms  for  the  Air  Force  and 
other  elements  of  the  Department  of  Defense  (DOD).  A  long  history 
of  successful  performance  in  this  broad  role  provides  MITRE  with 
detailed  knowledge  of  the  systems’  operational  capabilities  and  needs, 
proficiency  in  their  systems  engineering,  and  a  C’Trelared  corpomre 
memory  unmatched  by  any  other  organization.  That  background  is 
the  foundation  of  this  book  on  systems  engineering  at  MITRE. 

Chapter  2  describes  MITRE’s  typical  systems  engineering  contribu¬ 
tions  during  all  phases  of  major  C’l  acquisition  programs.  The  fol¬ 
lowing  chapters  summarize  the  factors  that  uniquely  qualify  MITRE 
for  the  C^I  systems  engineering  role  and  explore  some  special  topics 
in  systems  engineering.  The  material  emphasizes  the  importance  of 


systems  engineering  and  illustrates  approaches  and  techniques  that 
have  proven  successful  in  challenging  and  high-visibility  C*1 
programs. 

The  novice  and  the  experienced  systems  engineer,  as  well  as  others 
who  wish  to  better  understand  systems  engineering  as  practiced  by 
MITRE,  will  benefit  from  these  discussions.  There  are  special  insights 
on  how  system  requirements  are  established;  on  interactions  among 
the  systems  engineer,  government,  and  industry;  on  being  effective  in 
the  large,  complicated,  and  volatile  environment  of  a  major  acquisi¬ 
tion  program;  and  on  a  systems  engineering  approach  to  the  sophisti¬ 
cated  technologies  involved  in  CT  systems,  including  risk  reduction. 
Finally,  there  is  a  chapter  devoted  to  some  special  system  engineering 
activities  that  help  transform  an  idea  into  a  system,  and  more  pre¬ 
cisely,  one  that  satisfies  the  government’s  requirements  at  a  reason¬ 
able  cost  and  on  a  reasonable  schedule,  thereby  providing  an 
effective,  affordable  operational  capability  matched  to  an  important 
national  need. 

The  details  contained  in  this  book  derive  directly  from  MITRE’s 
work  for  the  Electronic  Systems  Center  (ESC)  of  the  Air  Force  Mate¬ 
riel  Command  (AFMC)  and  for  the  predecessors  of  those  organiza¬ 
tions,  especially  the  Electronic  Systems  Division  (ESD)  of  the  Air 
Force  Systems  Command  (AFSC).  They  reflect  MITRE’s  long  and 
special  relationship  with  ESC,  although  many  of  the  important  con¬ 
siderations  discussed  also  apply  to  systems  engineering  work  for 
other  government  agencies. 

MITRE  provides  systems  engineering  support  to  ESC  as  an  essen¬ 
tial  part  of  an  integrated  project  team  on  each  program.  The  empha¬ 
sis  placed  on  the  Corporation’s  activities  should  in  no  way  be 
construed  to  indicate  that  MITRE  does  not  appreciate  fully  the  essen¬ 
tial  roles  of  government  and  industry  in  achieving  required  system 
capabilities.  The  Corporation’s  role  is  to  work  between  government 
and  industry  as  a  catalyst,  an  honest  broker,  to  achieve  capabilities. 
Success  depends  on  a  cooperative,  mutually  respectful,  good  faith 
effort  by  all  three  parties. 


A  large  dose  of  common  sense 
is  0  very  valuable  systems 
engineering  tool.  One  will  be 
surprised  at  how  often  common 
sense  seems  to  be  missing. 


It  is  always  a  feeling  of  great 
pride  and  satisfaction  when  the 
importont  systems  one  has 
worked  so  hard  to  perfect 
perform  well  in  helping  to 
accomplish  the  operotionol 
mission  they  were  designed 
to  support. 


In  Its  systems  engineering  work,  MITRH's  main  concern  is  to  help 
CListonK’j  achieve  necessary  capabilities  on  a  reasonable  schedule  and 
r  asonable  cost.  In  a  paper  on  quality  assurance  at  .MU  RE,  presi¬ 
dent  Zraket’s  foreword  notes; 

.  .  .  success  of  the  MITRE  work  ts  explicitly  tied  to  the  needs  of 
our  clients  and  to  the  success  that  each  client  achieves  in 
satisfying  those  needs  with  development  programs  aci.]uired  at 
a  reasonable  cost  and  on  a  reasonable  schedule.  Although 
MITRE  does  not  have  ultimate  control  of  many  of  the  key  factors 
that  determine  the  success  of  a  program,  the  quality  of  the 
MITRE  system  engineering  on  a  given  system  is  judged,  in  part, 
on  how  well  the  system  satisfied  the  client's  needs.' 

.MITRE  believes  a  program  is  a  success  if  the  government  achieved 
the  best  capability  possible  for  the  time  and  money  invested.  It  is 
always  a  feeling  of  great  ptide  and  satisfaction  when  the  important 
systems  one  has  worked  so  hard  to  perfect  perform  well  in  helping  to 
accomplish  the  operational  mission  they  were  designed  to  support. 
MITRE’s  systems  engineering  work  provides  ample  opportunity  for 
achieving  this  professional  reward. 

Just  as  in  many  activities  where  experience  is  a  key  teacher,  and  espe¬ 
cially  for  those  that  have  some  attributes  of  an  art,  consummate  skill  at 
systems  engineering  cannot  be  achieved  through  study  alone.  This  book 
provides  some  help  to  those  involved  in  systems  engineering,  but  it 
cannot  substitute  for  actual  experience.  Likewise,  true  wisdom  often  is 
not  fully  appreciated  until  it  is  personally  experienced.  The  inexperienced 
systems  engineer  will  find  it  difficult  to  relate  to  the  importance  of  some 
of  the  material.  Much  of  it  will  seem  not  much  more  than  common 
sense.  It  is,  indeed,  just  that.  A  large  dose  of  common  sense  is  a  very 
valuable  systems  engineering  tool.  One  will  be  surprised  at  how  often 
common  sen^  seems  to  be  missing.  In  working  on  an  actual  acquisition 
program,  the  pressures  of  the  moment  can  make  people  reluctant  to  do 
what  seems  sensible,  especially  when  that  is  difficult  under  existing 
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'  Quality  Assurance  at  MITRE.  Vol.  1,  .\I88-,?9,  The  MITRt  Corpor,ition,  Bedford. 
MA,  Oetoher  1988,  n.  3. 


circumstances.  .Succcsshilly  working  through  the  conflicting  demands  of 
high  performance  on  short  schedules  and  at  low  costs,  in  an  environment 
where  many  of  the  key  personalities  and  controlling  factors  are  con¬ 
stantly  changing,  is  indeed  an  art.  It  is  best  practiced  by  a  skilled,  experi¬ 
enced,  and  dedicated  systems  engineering  team. 

For  less  experienced  systems  engineers,  the  material  serves  as  a 
guide  to  what  is  truly  important.  It  provides  a  useful  reference  for 
guidance  and  resolve  when  the  going  gets  tough.  For  experienced 
systems  engineers,  this  reiteration  of  what  is  important  will  reinforce 
their  determination  to  do  what  is  necessary,  however  difficult,  to  help 

customers  achieve  each  required  capabilitv  on  a  reasonable  schedule  ,,  .. 

The  extroordinory  systems 

and  at  reasonable  cost. 

■A  good  systems  engineer  becomes  better  through  detailed  knowl-  engineer  hos  the  necessary 

edge  of  both  the  operational  needs  and  the  technologies  involved.  A 

background  and  knowledge 

systems  engineer  should  have  some  understanding  of  the  theory  of 

systems  engineering  and  some  training  in  the  associated  techniques.  ond  con  opply  it  without 

Also,  a  systems  engineer  should  be  skilled  in  one  or  more  of  the 

,  ,  .  ,  ,  -re-  L  1  j  i.-  being  overly  preoccupied  by 

technologies  relevant  to  the  systems  of  interest.  This  book  identifies 

those  attributes  that  distinguish  the  exceptional  from  the  ordinary.  It  techniques 

focuses  on  the  unique  aspects  of  MlTRE’s  approach  to  systems  engi¬ 
neering.  The  extraordinary  systems  engineer  has  the  necessary  back¬ 
ground  and  knowledge  and  can  apply  it  without  being  overly 
preoccupied  by  the  techniques.  He  or  she  relates  well  to  the  environ¬ 
ment  within  which  the  system  is  being  developed  and  is  dedicated  to 
achieving  the  required  operational  mission  capability. 

At  this  point,  a  few  words  on  what  is  not  included  are  in  order.  A 
detailed,  step-by-step  process  for  systems  engineering  goes  beyond  the 
scope  of  this  book.  MITRE  is,  however,  currently  supporting  an  Air 
Force  effort  to  more  precisely  define  a  systems  engineering  life  cycle 
flow  process  to  be  used  in  future  acquisition  programs.  In  addition,  a 
detailed  “how  to”  primer  on  each  of  the  many  relevant  MITRE 
activities,  such  as  how  to  write  a  system  specification,  or  participate 
in  the  source  selection  process,  is  not  included  here.  Many  such 
papers  already  exist  and  are  described  briefly  in  the  Appendix. 


No  attempt  is  made  in  this  material  to  detail — and  certainly  not  to 
reform — the  DOD  approach  to  the  acquisition  of  C’l  systems.  Over 
the  years,  many  different  studies  have  effected  changes  to  that  pro¬ 
cess.  A  recent  Carnegie  Commission  study  proposes  a  completely  new 
approach  modeled  after  that  used  by  industry  in  major  commercial 
development  projects.’  While  striving  to  improve  the  DOD  acquisi¬ 
tion  process,  one  important  fact  must  be  remembered:  through  the 
acquisition  efforts  of  the  DOD  and  the  defense  industry,  there  are 
many  very  capable  military  systems  operational  today.  It  is  not  so 
much  the  detailed,  ideal  process  that  matters,  but  the  experience, 
skill,  and  good  judgment  with  which  the  people  involved  adapt  that 
process  to  the  particular  circumstances  and  program.  This  book 
echoes  a  sentiment  expressed  by  Arthur  D.  Hall: 

I  have  resisted  pandering  to  a  common  desire  for  more  treatment 
of  methods  or  tools,  on  the  ground  that  sensitivity,  knowledge, 
skill,  and  good  judgment  with  the  process  itself  are  the  most 
important  factors  in  success.  ’ 

Systems  Engineering — Destriptions  and  Definitions 

“  When  I  use  a  word,  ”  Humpty  Dumpty  said,  in  a  rather 
scornful  tone,  “it  means  what  I  choose  it  to  mean —  neither  more 
nor  less.  ” 

“The  question  is,"  said  Alice,  “whether  you  can  make  words 
mean  so  many  different  things.  ” 

“The  question  is,"  said  Humpty  Dumpty,  “which  is  to  be 
master — that’s  all.  ” 

Lewis  Carroll,  Through  the  Looking  Glass  (from  C.D.  Flagle, 
et  al."*) 

’  "A  R.idic;il  Reform  of  rhe  Defense  Acquisition  .System,"  Statement  of  the  Carnegie 
Ciommission  on  Science,  Technology,  and  Government  (New  York:  1  December 
1992). 

'  Arthur  D.  Hall,  Metasystems  Methodology  (New  York:  Pergamon  Press,  1989), 
p.  xiii. 

^  C.D.  Hagle,  W.H.  Huggins,  and  R.H.  Roy,  Operations  Research  and  System 
Engineering  (Baltimore;  The  Johns  Hopkins  Press,  1960),  p.  8. 


For  a  term  so  widely  used,  there  has  been  relatively  little  written 
about  systems  engineering  over  the  last  30  years.  As  noted  by  W.R. 
Beam,’  the  Library  of  Congress  holdings  for  1989  contained  only  213 
references  to  systems  engineering.  A  group  of  books  on  systems 
engineering  was  published  in  the  late  1950s  and  another  group  was 
published  in  1989  and  1990,  but  not  much  in  between.  It  is  interest¬ 
ing  that  all  of  these  texts  refer  to  “systems  engineering,”  with  the 
exception  of  R.E.  Machol's  book,*  which  discusses  “system  engineer¬ 
ing.”  Both  phrases  will  be  used  in  this  book.  MITRE’s  niche  is  sys¬ 
tems  engineering,  but  on  any  particular  program,  MITRE  is  the 
sys’^em  engineer  in  the  same  way  there  is  a  government  system  pro¬ 
gram  director  and,  perhaps,  a  system  contractor.  Use  of  the  phrase 
“systems  engineering”  recognizes  the  fact  that  every  system  the 
government  acquires  must  interface  with  many  other  existing  and 
planned  systems.  A  major  concern  in  MITRE’s  systems  engineering 
work  is  the  interoperability  among  all  the  systems  that  constitute 
what  is  referred  to  in  Chapter  2  as  an  operational  mission  capability. 
In  that  sense,  even  on  an  individual  program,  MITRE  is  performing 
systems  engineering. 

Most  textbooks  on  systems  engineering  suggest  that  the  first  for¬ 
mal  use  of  the  term  “system  engineering”  was  by  the  Bell  Telephone 
Laboratories  in  the  1940s.  The  early  texts  emphasize  a  multidisci- 
plined  team  as  the  essential  ingredient  of  successful  systems  engineer¬ 
ing.  They  suggest  that  such  an  approach  was  crucial  to  activities  of 
ancient  times,  such  as  the  construction  of  the  Egyptian  pyramids,  and 
more  recent  ones,  such  as  the  development  of  television  broadcast 
services.  The  necessity  of  a  multidisciplined  team  for  many  different 
undertakings — including  many  outside  the  field  of  systems  engineer¬ 
ing — is  almost  taken  for  granted  today.  Consistent  with  today’s  use 
of  integrated  product  teams,  MITRE’s  approach  to  systems  engineer¬ 
ing  has  always  involved  project  teams  consisting  of  people  with 

'W.R.  Beam.  Systems  Engineering  Architecture  and  Design  (New  York;  McGraw- 
Hill,  1990),  p.  X. 

'’R.E.  Machol,  System  Engineering  Eiandhook  (New  York:  McGraw-Hill,  1965). 


expertise  in  all  relevant  technologies  and  in  the  operational  areas  that 
the  system  is  to  support. 

All  the  books  on  systems  engineering  contain  discussions  that  are 
relevant  to  MITRE’s  work.  The  material  is  very  lengthy  and  tends  to 
bury  the  important  points  within  discussions  of  disciplines  such  as  opera¬ 
tions  research,  or  in  component  discussions  such  as  computers,  radars,  or 
communications  devices.  However,  some  quotations  from  the  earlier 
books  help  to  establish  what  systems  engineering  means.  They  reflect  the 
sort  of  thinking  that  pervaded  systems  engineering  at  MITRE  when  the 
Corporation  was  formed  and  remain  relevant  today. 

In  1960,  C.D.  Flagle  et  al.  noted: 

System  engineering  introduces  us  to  a  new  order  of  complexity 
in  that  a  system  depends  for  the  performance  of  its  assigned 
function  on  the  intimate  cooperation  of  a  number  of  devices, 
each  a  complicated  mechanism  in  itself.  It,  therefore,  accelerates 
the  obsolescence  of  the  cut  and  dried  empirical  methods  that 
formerly  characterized  all  the  useful  arts  and  that  still  play  a 
significant  role  in  them,  engineering  and  medicine  not  excepted. 

In  systems  engineering  even  more  than  in  other  arts,  understand¬ 
ing,  the  product  of  scientific  research,  is  the  catalyst  of  techno¬ 
logical  progress.  It  is  essential  for  survival  in  a  competitive 
world. ^ 

They  go  on  to  observe. 

The  successful  system  engineer  so  balances  the  attributes  of 
discrimination  and  association  that  he  recognizes  real  from 
trivial  differences,  suhstanUve  from  illusory  resemblances;  he  is 
lofty  in  concept;  knowledgeable  and  wise  in  selection;  critical, 
alert,  and  scientifically  exact  in  analysis;  and  in  synthesis, 
imaginative  in  plan,  meticidous  in  operation,  and  practical  in 
execution.’* 


Flagle,  p.  80. 
”  Flagle,  p.  80. 


They  also  note, 

In  particular,  it  [system  engineering!  is  a  field  where  action  must 
be  backed  up  by  scientific  understanding — it  is  too  complicated 
for  purely  empirical  approaches."* 

As  will  be  discussed  in  some  detail  later,  MITRE  staff  must  be 
informed  on  all  matters  that  potentially  impinge  on  the  chances  for 
program  success.  They  must  be  extremely  discriminating  in  identify¬ 
ing  those  that  are  important  and  in  suggesting  how  they  should  be 
handled.  To  do  so  requires  that  the  systems  engineering  team  include 
in-depth  expertise  on  all  the  relevant  disciplines — scientific,  engineer¬ 
ing  and  management.  And  of  utmost  importance,  the  MITRE  staff 
must  have  a  genuine  understanding  of,  and  appreciation  for,  the 
capability  that  the  government  user  needs. 

In  his  paper  published  in  1957,  E.W.  Engstrom  wrote: 

Inevitably,  the  proliferation  of  “black  boxes”  brought  prob¬ 
lems  of  establishing  proper  interaction  among  them.  Thus,  the 
system  engineer  began  to  come  into  his  own,  performing  the 
essential  task  of  looking  ahead  to  the  ultimate  objective — the 
system — and  considering  the  whole  of  which  each  “black  box  ” 
formed  a  part.'^ 

With  this  statement  of  the  genesis  of  systems  engineering, 
Engstrom  went  on  to  characterize  it  in  a  way  that  very  much  applies 
to  MITRE’s  work  today: 

As  a  discipline,  the  systems  approach  has  these  characteristics  in 
all  cases,  regardless  of  the  great  variety  of  objectives  and  detailed 
engineering  methods: 

1 .  It  is  broad  in  scope,  ignoring  the  boundaries  that  separate  the 
various  academic  disciplines,  that  separate  research  from  engi¬ 
neering,  and  advanced  development  from  product  design  and 
marketing. 


’’  Flagle,  p.  68. 
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ing,  Vol.  76,  No.  2,  February  1957,  p.  113. 


2.  It  is  co-operative,  usually  involving  large  numbers  of  people 
and  functions  that  may  appear  at  first  glance  to  have  little  to  do 
with  one  another. 

3.  It  requires  compromise,  because  success  in  developing  a 
complex  system  normally  involves  a  sacrifice  at  one  or  more 
points  of  detail  for  the  sake  of  the  whole  system. 

4.  It  is  thorough — and  by  the  same  token  a  bit  skeptical.  The 
systems  engineer  must  examine  every  detail  that  bears  upon  the 
function  of  the  complete  system.  At  the  same  time,  he  must 
distrust  the  tempting  “easy"  solution  that  first  appears." 

Thorough  with  a  dose  of  skepticism,  transcending  all  relevant 
factors,  and  ready  to  compromise  as  necessary  to  achieve  the  neces¬ 
sary  system  capability — this  characterizes  well  the  challenge  faced  by 
MITRE  in  its  systems  engineering  tasks. 

A  review  of  the  literature  provides  many  different  definitions  of  sys¬ 
tem  and  system  engineering.  For  example,  R.E.  Machol  offers  several 
different  definitions  of  each  as  gathered  from  other  texts  on  the  subject.'^ 
They  range  from  mathematical  set-theoretic  definitions  to  rather  simplis¬ 
tic  ones.  DOD  documentation  defines  system  engineering  as 

the  application  of  scientific  and  engineering  efforts  to  (a)  trans¬ 
form  an  operational  need  into  a  description  of  system  perfor¬ 
mance  parameters  and  a  system  configuration  through  the  use 
of  an  iterative  process  of  definition,  synthesis,  analysis,  design, 
test,  and  evaluation;  (b)  integrate  related  technical  parameters 
and  ensure  compatibility  of  all  physical,  functional,  and  pro¬ 
gram  interfaces  in  a  manner  that  optimizes  the  total  system 
definition  and  design;  (c)  integrate  reliability,  maintainability, 
safety,  survivability,  human  engineering,  and  other  such  factors 
into  the  total  engineering  effort  to  meet  cost,  schedule,  support- 
ability,  and  technical  pe-  ormance  objectives." 

"  E.W.  Engstrom,  pp.  113-114. 

R.E.  .Machol,  System  Engineering  Handbook  (New  York;  McGraw-Hill,  1965), 

pp.  1-12. 

Engineering  Management,  DOD  M1L-STD-499A,  May  1974. 


Alternatively,  the  Defense  System  Management  College  offers  the 
following; 

Systems  engineering  is  the  management  function  which  controls 
the  total  system  development  effort  for  the  purpose  of  achieving 
an  optimum  balance  of  all  system  elements.  It  is  a  process  which 
transforms  an  operational  need  into  a  description  of  system 
parameters  and  integrates  those  parameters  to  optimize  the 
overall  system  effectiveness.^'* 

One  sample  of  the  definitions  used  by  industry  is  this  one  from 
IBM,  in  which  system  engineering  is 

the  iterative  but  controlled  process  in  which  user  needs  are 
understood  and  evolved  through  increasingly  detailed  levels  of 
requirements  specification  and  system  design  to  an  operational 
system;  includes  the  intellectual  control  and  integration  of  all 
disciplines  throughout  the  system  life  cycle  in  a  manner  so  as  to 
ensure  that  all  user  requirements  are  satisfied.'^ 

MITRE  has  no  corporate  definition  of  systems  engineering,  and  this 
book  will  not  provide  one.  Some  will  be  disappointed  at  that,  but  brief 
definitions  inadequately  convey  the  breadth  and  depth  of  the  systems 
engineering  activities  performed  by  MITRE.  On  the  other  hand,  MITRE 
has  been  instrumental  in  developing  .any  of  the  concepts  that  have 
become  an  integral  part  of  systems  engineering  as  it  is  practiced  today, 
both  at  MITRE  and  at  other  organizations.  Many  of  them  are  discussed 
in  detail  later.  From  the  beginning,  MITRE  recognized  that  the  systems 
engineer  must  actively  participate  in  system  design  evaluation,  and  con¬ 
duct  necessary  design  verification  activities.  Other  innovative  concepts 
include  the  use  of  the  system  itself  to  monitor  its  own  performance,’^  and 
to  provide  for  operator  training.  MITRE  also  recognized  the  need  for  a 

'■*  System  Engineering  Management  Guide,  Defense  System  Management  College, 
January  1990. 

"  System  Engineering  Principles  and  Practices,  IBM  Federal  Systems  Division, 
Berhesda,  MD,  June  1983. 

J.W.  Shay,  MITRE  System  Engineering  Book  Outline,  W-07353/0000/01,  The 
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system  to  continue  to  operate  in  degraded  conditions  resulting  from 
enemy  action  or  from  internal  system  failures.  The  use  of  live  exercises 
for  system  evaluation  and  the  appropriate  use  of  simulation  tools  for 
estimating  system  performance  were  early  MITRE  initiatives.''* 

The  phased  implementation  of  systems  such  as  the  Semi-Automatic 
Ground  Environment  (SAGE)  system  and  the  425L  NORAD  Com¬ 
mand  Operations  Center  in  the  late  1950s  and  early  1960s  is  an  early 
example  of  a  xMlTRE  approach  that  was  later  adopted  by  the  acquisi¬ 
tion  community  at  large  and  referred  to  as  evolutionary  development 
or  preplanned  product  improvement.  The  utility  of  an  operational 
employment  plan  in  managing  a  successful  acquisition  program  was  a 
MITRE  initiative. 

As  another  example  of  MITRE’s  contribution  to  the  art  of  systems 
engineering,  consider  the  following.  In  a  speech  to  the  Operations 
Research  Society  in  1961,  John  F.  Jacobs  of  MITRE  identified  the 


'*  J.F.  Jacobs,  Practical  Evaluation  of  Command  and  Control  Systems.  MTP  '^,  The 
■MITRE  Corporation,  Bedford,  MA,  November  1965. 


hierarchy  of  design  within  which  a  system  must  be  considered  and 
discussed  the  component,  subsystem,  and  system  levels.'*'  Beyond 
that,  he  explicitly  recognized  that  each  system  being  acquired  is 
imbedded  in  successively  higher  level  systems  that  address  individual 
mission  capabilities,  overall  military  systems,  and  ultimately  national 
systems  that  transcend  even  those  overall  systems.  For  example,  a 
radar  might  be  considered  a  component  of  an  air  defense  system,  air 
defense  part  of  the  nation’s  defense  system,  which  in  turn  is  part  of 
the  Joint  Chiefs  of  Staff’s  managed  military  system.  That  system  then 
must  operate  within  a  national  system  that  includes  others  such  as 
the  air  traffic  control  system.  Expanded  versions  of  these  concepts 
were  presented  by  Jacobs  to  the  Air  Staff  in  June  1962-*’  and  again  in 
1963.-' 

In  1976,  another  MITRE  paper  written  by  Jacobs  described  the 
concept  of  a  “system-of-systems.”^^  This  concept  explicitly  recognizes 
that  each  system  being  acquired  must  interoperate  with  others  at 
parallel  or  lower  levels,  and  that  the  collection  of  such  systems  is,  in 
turn,  embedded  in  other  systems  of  greater  scope.  At  that  time,  some 
people  even  used  the  term  “super  system  engineer”  to  describe  the  Air 
Force’s  need  for  systems  engineering — not  just  system  engineering — 
because  most  operational  mission  capabilities  actually  consist  of 
many  different  subcapabilities,  each  of  which  is  accomplished  by  an 
individual  system. 

The  remainder  of  this  book  addresses  the  specific  MITRE  activities 
that  constitute  its  role  as  systems  engineer  on  a  major  Air  Force  C^I 
system  acquisition  program.  All  are  important,  even  if  some  of  them 
lie  outside  a  precise  definition  of  systems  engineering.  When  the  term 
“systems  engineering”  is  used  here,  it  is  meant  to  include  all  of  them. 

'’J-F.  Jacobs,  Air  Force  Command  and  Control  System  Development,  SR-23,  The 
.MITRE  Corporation,  Bedford,  MA,  July  1961. 

^“j.F.  Jacobs,  The  Integration  and  Standardization  of  Automated  Information 
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Bedford,  MA,  August  1963. 

^^J.F.  Jacobs,  System  Architecture  and  System-of-Systems,  M76-206,  The  MITRE 
Corporation,  Bedford,  MA,  June  1976. 


As  systems  engineer,  MITRE  must  be  co^’nizant  of  a  range  of  factors 
that  transcend  the  strictly  technical.  These  factors  are  continuously 
assessed  to  determine  whether  they  indicate  that  some  action  should 
be  taken  to  improve  the  chances  of  achieving  the  needed  capability  on 
a  reasonable  schedule  and  at  reasonable  cost.  When  MITRE  project 
personnel  recognize  that  something  must  be  done,  it  is  incumbent  on 
them  to  work  with  the  government  program  director  and  the  inte¬ 
grated  product  team  to  make  sure  that  appropriate  action  is  taken. 

There  is  no  inference  that  the  implied  definition  of  systems  engi¬ 
neering  at  MITRE  would  apply  anywhere  else,  in  government  or  in 
industry.  Neither  is  it  meant  to  imply  that  MITRE  does  all  the  sys¬ 
tems  engineering  on  programs  for  which  it  is  assigned  the  systems 
engineering  role.  Indeed,  many  of  the  government  and  industry  activi¬ 
ties  on  a  program  are  substantial,  and  may  properly  be  referred  to  as 
systems  engineering. 
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mitre's  Role  in  Acquisition  Programs 

As  systems  engineer  on  a  C^I  system  acquisition  program, 
MITRE  makes  major  contributions  in  each  of  the  typical  program 
phases  from  initial  conception,  through  acquisition,  to  the  full 
operation  of  the  system  by  the  using  command.  MITRE’s  role, 
responsibilities  and  products  in  each  phase  are  the  subjects  of  this 
chapter.  This  chapter  also  poses  the  key  questions  that  must  be 
answered  and  discusses  MITRE’s  work  in  helping  to  answ-r  them. 
In  addition,  it  describes  the  value  MITRE  adds  in  the  systems 
engineering  role. 

Phases  of  on  Acquisition  Program 

Understanding  the  major  program  phases  through  which  each  new 
DOD  acquisition  program  moves  is  helpful  as  background  for  the 
discussions  of  MITRE’s  contributions.  This  section  describes  each 
phase  and  the  critical  times  when  decisions  must  be  made  before 
entering  the  next  phase.  Over  time  this  process  has  changed,  and  it 
will  continue  to  evolve  as  government  and  industry  work  to  improve 
the  acquisition  process. 


As  described  in  Program  Baselines  and  Milestones'^  and  more 
recently  in  Defense  Acquisition  Management  Policies  and  Procedures,'* 
in  a  classic  acquisition  program  the  government  recognizes  an  opera¬ 
tional  need  and  studies  potential  alternative  systems  that  might  provide 
the  required  capability.  If  the  need  is  ratified  and  one  or  more  of  the 
alternative  systems  is  judged  responsive  to  the  need,  affordable,  and 
obtainable  on  the  required  schedule,  the  system  is  approved  for  a  phase 
in  which  industry  is  given  a  contract  to  build  this  system.  The  first 
major  phase.  Phase  0,  is  one  in  which  needs  are  assessed,  alternative 
system  solutions  studied,  and  decisions  made  on  whether  to  proceed 
with  actual  procurement.  To  describe  this  phase  further,  there  is  a 
decision  point  known  as  Milestone  0  at  which  a  proposed  program 
need  is  approved  or  disapproved  for  entry  into  the  next  sequential 
activity.  Concept  Exploration  and  Definition.  Some  of  the  factors 
considered  include  alternative  programs,  associated  costs  and  sched¬ 
ules,  long-term  affordability  (sometimes  referred  to  as  life-cycle  cost) 
and  the  proposed  acquisition  strategy.  At  the  end  of  that  phase,  the 
program  reaches  Milestone  1.  Then,  the  program  is  evaluated  to  deter¬ 
mine  whether  to  proceed  to  the  next  step  in  the  acquisition  process,  the 
Demonstration  and  Validation  phase.  This  is  also  referred  to  as  Phase 
I.  In  Phase  I,  risk  reduction  efforts  are  performed,  critical  technologies 
demonstrated,  and  refined  performance  objectives,  system  concepts, 
costs,  and  schedules  developed. 

After  completing  the  Demonstration  and  Validation  phase,  the 
Milestone  II  decision  determines  whether  to  proceed  into  full-scale 
system  development.  If  approved,  the  program  enters  Phase  II,  Engi¬ 
neering  and  Manufacturing  Development.  In  this  phase,  profit-mak¬ 
ing  industry  that  may  have  been  involved  in  the  earlier  studies  and 
analyses  builds  the  development  system.  The  Milestone  11  decision 
might  also  include  approval  for  low-rate  production  of  the  proposed 
system  to  verify  that  the  contractor  has  the  knowledge  and  facilities 

Program  Baselines  and  Milestones,  Defense  Systems  Management  College, 
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(•or  full-scale  production  of  many  systems.  It  is  one  challenge  to  build 
a  single  copy  of  a  system  that  performs  acceptably;  it  is  quite  another 
to  effectively  replicate  the  system  many  times,  as  is  often  the  require¬ 
ment  in  Cd  programs. 

At  Milestone  III,  the  government  reviews  what  was  accomplished 
during  the  development  phase  and  decides  whether  to  proceed  into 
Phase  HI,  Production  and  Deployment,  in  which  full  production  and 
initial  deployment  to  the  forces  that  will  operate  the  system  take 
place.  In  addition  to  the  results  of  the  development  program,  other 
factors  considered  include  a  current  estimate  of  the  threat,  probable 
production  and  life-cycle  costs,  likely  schedules,  reliability  and  main¬ 
tainability,  logistics  supportability,  prodncibility,  and  many  others. 

When  the  production  system  is  built  and  tested  to  demonstrate 
that  the  contractor  provided  the  system  promised  in  its  contract,  the 
program  moves  into  a  government  operational  test  and  evaluation 
period  in  which  the  system’s  operational  effectiveness  is  assessed. 
Phase  IV,  Operations  and  Support,  overlaps  Phase  III.  After  the 
system  has  been  in  operation  for  some  time,  and  on  an  as-required 
basis,  a  Milestone  IV  decision  may  occur.  At  that  time,  the  necessity 
for  a  major  system  upgrade  is  reviewed.  If  one  is  approved,  and  on  an 
as-required  basis,  changes  are  made  to  those  systems  that  are  still  in 
production.  Major  ch.a.nges  to  the  systems  that  have  been  already 
been  produced  by  the  time  of  Milestone  IV  compete  with  other  pos¬ 
sible  alternatives  in  a  new  Phase  0. 

Before  describing  MITRE’s  systems  engineering  activities  in  each 
phase  of  a  C’l  system  acquisition  program,  there  is  one  note  of  cau¬ 
tion.  The  above  description  should  not  lull  anyone  into  believing  that 
the  acquisition  process  is  as  straightforward  as  it  is  represented  here. 
As  discussed  in  Chapter  3,  the  acquisition  environment  is  much  more 
complicated  than  the  ideal  described. 

Corporate  Role  and  ResponsibiTrties 

MITRE’s  role  is  to  help  the  government  achieve  required  capabili¬ 
ties.  In  that  role,  the  Corporation  often  participates  in  studies  and 
analyses  leading  to  the  establishment  of  an  acquisition  program. 


pertornis  as  the  system  engineer  during  the  acquisition,  and  partici¬ 
pates  in  the  evaluation  of  system  performance  after  the  system  be¬ 
comes  operational.  The  involvement  from  early  conceptual  studies 
through  acquisition  and  system  operation  provides  MITRE  with 
unique  insight  into  the  government’s  operational  requirements  and 
the  performance  that  is  achieved  in  field  operation.  Understanding 
the  requirements  helps  in  making  judgments  and  recommendations  as 
systems  engineer.  Observing  actual  system  performance  is  an  impor¬ 
tant  foundation  for  subsequent  analysis  of  future  government  needs. 
A  history  of  having  performed  successfully  in  this  broad  role  on  .Air 
Force  C  ’l  systems  for  over  35  years  provides  .MITRE  with  in-depth 
knowledge  of  operational  needs,  systems  engineering  skill,  and  a 
corporate  memory  unmatched  by  any  other  organization. 
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This  chapter  summarizes  the  types  of  MITRE  activities  that  may  occur 
prior  to  the  establishment  of  an  acquisition  program  by  the  government. 
The  Corporation’s  activities  in  each  of  the  acquisition  program  phases 
are  reviewed  and  its  work  in  helping  to  evaluate  the  pierformance  of  the 
operational  system  is  described.  Although  no  step-by-step  prescription  is 
provided,  important  activities  are  identified,  crucial  questions  to  be 
answered  are  posed,  and  some  cautions  and  suggestions  are  discussed. 


The  chapter  is  organized  hy  discrete  acquisition  program  phase. 
However,  in  reality,  programs  are  otten  in  several  ditferent  phases  at 
the  same  time.  For  example,  at  any  time,  there  is  an  existing  opera¬ 
tional  capability  on  which  the  new  acquisition  program  will  be  based 
While  a  new  capability  is  in  development  and  test,  the  next  generation 
ot  that  capability  may  be  in  early  conceptual  planning.  The  evolution¬ 
ary  nature  of  C'l  and  other  systems  is  an  important  consideration  in 
planning  for  operational  capabilities  and  in  carrying  out  the  associated 
acquisition  programs.  Even  though  the  section  emphasizes  activities 
characteristic  of  a  new  acquisition  program,  it  should  be  n.rted  that 
most  MITRE  work  involves  systems  engineering  for  major  additions  to 
existing  capabilities.  The  additions  are  acquired  by  the  same  program 
office  that  acquired  the  original  capability,  and  the  Ciorporation  nor¬ 
mally  continues  as  systems  engineer  throughout  the  program. 

Conteptuol  Planning 

A  quotation  from  Shakespeare,  as  noted  by  A.D.  Hall,  helps  to 
characterize  conceptual  planning  for  new  systems.-' 

When  first  tee  mean  to  build. 

We  first  survey  the  plot,  then  draw  a  model; 

And  when  we  see  the  figure  of  the  house. 

Then  we  must  rate  the  cost  of  erection; 

Which  if  we  find  outweighs  ability. 

What  we  do  we  then  hut  draw  anew  the  model 
In  fewer  offices,  or  at  least  desist 
To  build  at  alii 

Shakespeare,  King  Henry  IV,  Part  2,  Act  1,  Scene  3,  Line  4 

Many  activities,  sometimes  over  a  number  of  years,  precede  the 
government’s  establishment  of  a  new  acquisition  program  or  a  major 
initiative  under  an  existing  program.  The  first  impetus  to  establish  a 
new  program  may  come  from  a  newly  perceived  user  requirement, 
change  in  enemy  threat,  availability  of  more  capable  technology,  or  an 
opportunity  to  achieve  required  system  performance  at  substantially 
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lower  costs.  Suggestions  for  possible  acquisition  programs  may  come 
from  any  sector  of  government  and  industry,  including  MITRE.  How¬ 
ever,  the  user  community,  in  conjunction  with  the  DOD,  in  the  end 
establishes  the  requirements  in  any  given  case.  The  DOD  and  Congress 
provide  for  the  establishment  and  funding  of  an  acquisition  program. 
What  constitutes  a  system  requirement  and  how  one  is  established 
deserve  further  elaboration  here. 

Just  as  'here  is  no  universally  accepted  definition  of  “system”  or 
“system  engineering”  in  the  C^I  acquisition  business,  there  is  no 
unanimity  of  opinion  on  what  level  of  information  constitutes  a 
legitimate  description  of  a  user  requirement.  Some  people  suggest  that 
the  use  of  the  word  “requirement”  is  harmful  because  it  tends  to 
connote  something  that  is  inviolate,  even  when  circumstances  dictate 
that  it  is  no  longer  practical  or  necessary.  Everyone  would  agree  that 
a  requirements  statement  such  as  “defend  the  United  States”  is  an 
inadequate  prescription  to  permit  a  responsive  system  to  be  built.  At 
the  other  end  of  the  spectrum,  most  would  also  agree  that  it  is  inap¬ 
propriate  for  a  user  requirements  statement  to  specifically  identify  the 
hardware  to  be  acquired. 

In  a  paper  on  defense  planning,  General  Glenn  Kent  (U.S.  Air  Force 
retired)  suggests  that  it  may  be  legitimate  to  say  there  is  a  requirement 
to  increase  the  nation’s  capability  to  achieve  some  operational  objec¬ 
tive.  He  notes  that  one  should  be  careful  in  suggesting  that  there  is  a 
requirement  to  achieve  some  particular  operational  task.  He  then  goes 
on  to  state,  “We  should  not  say  that  we  have  a  requirement  for  a 
particular  weapon  or  system,  and,  then,  we  have  a  requirement  for 
certain  performance  features  in  that  system.”  As  examples  of  what  is 
meant  by  “operational  objective”  and  by  “operational  task,”  General 
Kent  describes  preventing  the  Soviet  Union  from  dominating  Western 
Europe  as  a  national  objective  that  involves  strategies,  such  as  provid¬ 
ing  a  robust  forward  defense  with  conventional  forces.  This  strategy  in 
turn,  he  notes,  might  involve  operational  objectives,  such  as  delaying/ 
damaging  of  Soviet  follow-on  forces.  That  objective  could  involve  a 
number  of  tasks,  such  as  damaging  bridges.  General  Kent  suggests  that 


The  effectiveness  end  cost 
studies  implied  in  choosing 
among  operotionol  objectives 
ond  in  selecting  systems  to 
perform  those  tasks,  requite 
the  technical  expertise  of  the 
development  community. 


it  may  be  appropriate  to  refer  to  “requirements”  down  to  the  opera¬ 
tional  task  level,  but  not  beyond  that,  into  the  realm  of  systems, 
subsystems,  and  hardware.’^  That  is.  General  Kent  believes  it  may  be 
appropriate  to  establish  a  requirement  for  destroying  bridges,  but  not 
for  specifically  how  that  will  be  done.  He  also  states  that  choosing 
among  the  alternative  ways  to  achieve  a  national  objective,  i.e.,  among 
operational  objectives,  requires  study  of  the  effectiveness  and  cost  of 
doing  the  tasks  implied  by  the  operational  objective.  That  in  turn 
requires  analysis  of  the  alternative  systems  that  might  help  perform  the 
various  tasks. 

The  effectiveness  and  cost  studies  implied  in  choosing  among 
operational  objectives  and  in  selecting  systems  to  perform  those  tasks, 
require  the  technical  expertise  of  the  development  community.  In 
particular,  MITRE  has  an  important  role  to  play  in  these  activities. 

In  this  book,  the  word  “capability”  is  used  interchangeably  with 
the  phrases  “mission  capability”  or  “mission  system  capability.” 

They  are  meant  to  be  equivalent  to  General  Kent’s  “operational 
objective.”  The  individual  system  that  results  from  an  acquisition 
program  is  not  necessarily  a  capability,  as  the  word  is  used  here. 

Many  parrs  of  the  government  and  nongovernment  groups  contribute 
to  the  process  of  establishing  system  requirements.  By  actively  participat¬ 
ing  in  the  analysis  of  alternative  operational  objectives  and  tasks,  MITRE 
can  help  to  establish  meaningful  and  feasible  requirements.  However,  in 
the  end,  the  ultimate  using  command,  in  conjunction  with  its  service  and 
DOD  headquarters,  establishes  the  requirements  for  any  particular 
acquisition  program.  The  development  agencies  then  must  satisfy  those 
requirements,  although  funding  and  scheduling  constraints  may  result  in 
continuing  reassessment  of  the  requirements. 

Experience  indicates  that  there  are  few'  absolutes  in  the  establish¬ 
ment  of  user  requirements  for  capabilities  that  are  to  be  satisfied  by 
the  acquisition  of  a  large,  complicated  command  and  control  sys¬ 
tem.  One  might  imagine  that  users  know  exactly  what  capability 
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they  wish  to  achieve  and  can  describe  it  succinctly  to  the  agency 
assigned  to  provide  it  to  them.  The  real-life  situation  is  considerably 
different.  In  a  recent  book  on  systems  engineering.  Beam  makes  the 
following  comments  on  the  process  by  which  user  requirements  are 
established: 

System  requirements  may  emerge  from  a  user  organization,  but 
even  here  they  may  represent  the  unresolved  views  of  a  group  of 
individuals.  Even  at  best,  users  are  generally  not  certain  just 
what  they  really  need.  This  is  especially  true  if  (a)  they  have  had 
no  experience  with  similar  systems,  and  (b)  no  such  systems  are 
currently  available.  Thus: 

•  Requirements  will  in  most  cases  be  fragmentary,  at  least  in 
some  respects. 

22  •  Requirements  will  usually  address  high-level  systems  needs, 

but  key  requirements  may  be  omitted  from  attention. 

•  System  requirements  are  often  overstated,  beyond  actual 
needs,  because  special  groups  of  users  pose  their  own  require¬ 
ments  independently  without  overall  control  within  the 
user  organization. 

•  Fine  detail  requirements  are  often  included  among  true  top- 
level  ones. 

•  Seldom  are  requirements  prioritized;  users  are  always  hope¬ 
ful  that  they  will  achieve  every  need  or  wish. 

•  It  is  not  uncommon  to  find  sets  of  inconsistent  requirements 
for  any  large  or  complex  system. 

•  Some  requirements  may  be  given  whether  really  needed  or 
not,  and  some  may  be  given  merely  because  it  has  become 
traditional  to  ask  for  them.  This  is  often  true  where  there  are 
documented  standards,  since  naming  a  standard  document  is 
often  simpler  than  deciding  what  is  truly  required.^^ 

^'W.R.  Beam,  Systems  Engineering  Architecture  and  Design  (New  York:  McGraw- 
Hill,  1990),  p.  54. 


A  using  command  is  made  of  many  different  people  with  different 
and  valid  concerns — operations,  maintenance,  personnel,  training, 
and  logistics.  There  is  great  variety  in  the  experience  of  these  people 
and  in  the  relevance  of  that  experience  to  the  particular  acquisition 
being  planned.  Most  of  them  have  challenging,  full-time  jobs  associ¬ 
ated  with  carrying  out  the  current  mission  of  the  command,  and 
those  jobs  take  priority.  Describing  a  new  capability  is  not  an  easy 
thing  to  do.  Evaluating  alternative  operational  objectives  and  tasks  is 
time-consuming  and  demands  considerable  technical  expertise.  It  is 
especially  difficult  to  communicate  user  requirements  to  people  in  the 
development  community  who  do  not  have  the  same  experience  and 
understanding  as  the  user.  With  little  time  to  think  about  it,  and  in 
many  cases,  with  limited  directly  applicable  operational  experience,  it 
is  understandable  that  some  user  representatives  have  difficulty  in 
describing  the  desired  capabilities  to  development  agencies.  It  should 
also  not  come  as  any  surprise  that  different  levels  within  an  operating 
command  may  have  different  priorities  among  various  requirements, 
or  indeed,  very  different  requirements.  In  particular,  a  commander  is 
apt  to  have  much  greater  insight  into  the  command’s  most  urgent 
needs,  and  much  more  conviction  on  how  those  needs  should  be 
fulfilled,  than  people  at  lower  levels  in  the  using  organization. 

To  complicate  matters,  proposed  requirements  for  a  new  capability 
must  also  be  reviewed  and  approved  by  higher  headquarters  within 
the  user  command  service,  negotiated  with  the  other  services  with 
which  the  capability  interacts,  and  DOD  must  agree  with  them  and 
approve  their  funding.  Congress  and  the  congressional  staff  are  often 
involved  in  establishing  and  reviewing  a  program.  Getting  agreement 
is  not  easy.  Compromise  is  necessary  and  the  potential  implications 
on  the  subsequent  acquisition  program  are  often  very  significant. 

The  system  acquisition  process  used  by  the  government  also  com¬ 
plicates  the  process  of  establishing  requirements  and  inhibits  effective 
achievement  of  required  mission  system  capabilities.  In  the  late  1950s 
and  early  1960s,  the  agencies  responsible  for  acquisition  programs, 
known  as  system  programs  offices  (SPOs),  had  broader  responsibili- 


ties  ihan  they  typically  do  today.  The  SAGE  SPO  was  responsible  for 
acquiring  all  continental  air  defense  system  sensors,  communications, 
and  control  centers,  and  for  ensuring  that  they  interfaced  properly 
with  the  weapons  systems  acquired  separately  by  other  Air  Force  and 
Army  program  offices.  The  407L  Tactical  Air  Control  SPO  had 
similar  responsibility  for  the  Air  Force’s  tactical  air  control  system. 
MITRE’s  first  system  engineering  job  was  for  SAGE;  the  Corporation 
also  had  system  engineering  responsibility  for  407L.  Over  time,  the 
approach  of  using  “basket  SPOs”  fell  into  disfavor  as  the  resulting 
programs  became  very  large  and  costly,  and  those  unfamiliar  with 
program  details  found  it  difficult  to  determine  what  was  actually 
being  done  and  how  well. 

As  an  alternative  to  the  basket  SPO  approach  for  developing  sys¬ 
tem  capabilities  such  as  air  defense,  tactical  air  operations,  or  finding 
and  killing  ground  targets,  the  capabilities  were  broken  down  further 
into  their  major  subsystems;  these  subsystems  were  acquired  by 
separate  government  program  offices.  With  this  change,  a  govern¬ 
ment  program  director  was  assigned  the  job  of  buying  a  radar,  or  a 
piece  of  communications  equipment,  or  an  operations  center.  Direc¬ 
tion  for  each  program  tended  to  become  very  bounded:  do  precisely 
this,  in  this  much  time  and  for  this  much  money.  The  program  direc¬ 
tion,  time,  and  money  rarely  provided  for  the  activities  to  interface 
the  various  subsystems.  No  rea)  aurntion  was  given  in  the  individual 
acquisition  programs  to  making  sure  that  all  the  activities  would 
converge  in  a  way  that  would  provide  the  overall  mission  capability 
that  the  users  demanded. 

What  was  intended  as  a  good  business  practice  actually  introduced 
serious  problems  into  the  process  of  achieving  a  capability.  In  this 
paper,  a  radar  is  not  a  mission  capability — or  as  used  here — not  a 
capability.  In  the  language  of  this  book,  the  government  began  a 
process  of  buying  subsystems.  To  achieve  an  operational  mission 
capability,  each  subsystem  has  to  interoperate  with  other  existing  and 
planned  subsystems,  and  the  entire  complement  of  subsystems  must 
be  orchestrated  by  available  operating  and  maintenance  personnel. 
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The  new  aceiuisition  approach  teiuleci  to  delay  recognition  of  impor¬ 
tant  i''reractions  among  the  suhsystems  until  they  hegati  field  opera¬ 
tion.  Of  course,  the  user  community  properly  complained  when  they 
did  not  get  the  capability  they  needed.  In  some  cases,  acquisition 
programs  were  deemed  failures  when  in  part  the  problem  w  as  a  result 
of  the  acquisition  process  itself.  user/developer  tension  in  the  acqui¬ 
sition  process  developed  and  remains  a  cause  of  some  of  the  problems 
that  still  beset  (i'l  acquisition  programs  toda\ .  Users  try  to  specify 
technical  solutions;  developers  question  operational  tieeds.  Both  sides 
need  to  do  a  better  job  in  their  own  domains. 

In  man)  cases  today,  there  is  no  overall  architect  for  development 
of  a  mission  capability.  In  effect,  the  problem  of  assembling  the 
various  acquired  pieces  into  a  mission  capability  is  left  as  an  exercise 
to  the  ultimate  using  command,  often  without  access  to  the  technical 
resources  required  to  do  so.  Kach  contractor  is  obligated  to  deliver 
the  subsystem  described  in  the  contract,  each  program  director  to 
deliver  the  sidrsystem  ilescribed  in  the  program  direction.  Uertainly. 
both  groups  feel  a  responsibility  to  see  to  it  that  their  particular  piece 
fits  into  the  mission  capability.  However,  neither  has  the  responsibil¬ 
ity,  experience,  or  resources  to  ensure  that  a  mission  capability  is 
achieved.  .\HTRK,  however,  does  have  some  of  the  experience  neces- 
sarv  to  address  the  programs  in  terms  of  a  mission  capability.  Inter¬ 
estingly  enough,  the  change  in  acquisition  approach  increased  the 


government’s  need  for  a  group  with  the  experience  and  knowledge  to 
help  discrete  subsystems  function  together  as  capabilities.  In  this 
sense,  the  new  acquisition  approach  increased  the  government’s 
reliance  on  MITRE,  and  the  challenge  was  entirely  consistent  with 
the  Corporation’s  role  and  objectives. 

As  noted  above,  MITRE’s  first  job  was  to  help  the  Air  Force 
achieve  a  continental  air  defense  capability.  MITRE  has  had  systems 
engineering  responsibility  for  many  of  the  CM  related  subsystems  that 
are  essential  to  the  user’s  capabilities.  In  many  mission  areas,  the 
Corporation  has  had  that  responsibility  for  over  35  years.  MITRE 
therefore  has  the  inclination,  detailed  knowledge,  and  experience 
necessary  to  assume  a  broader  role  in  helping  the  ultimate  user 
achieve  the  required  capabilities.  With  the  necessary  knowledge 
comes  both  opportunity  and  responsibility.  Long  experience  with 
mission  capabilities  is  one  of  the  essential  factors  that  helps  define  the 
uniqueness  of  MITRE.  Again,  however,  the  opportunity  reflects  itself 
in  a  responsibility  to  help  each  customer  achieve  the  required  capabil¬ 
ity.  This  responsibility  is  recognized  and  welcomed  by  the  Corpora¬ 
tion.  It  is  taken  seriously  by  the  staff  and  management. 

The  using  command  has  the  ultimate  responsibility  for  achieving 
the  required  operational  mission  capability.  The  command  must 
define  what  it  needs,  participate  actively  in  the  process  of  establishing 
and  completing  the  necessary  acquisition  programs,  accept  the  new 
systems  and  combine  them  with  existing  systems,  and  provide  the 
personnel  and  training  to  operate  the  systems.  Much  has  been  written 
about  the  need  for  user  participation  in  C^I  acquisition  programs  if 
they  are  to  be  successful.  Their  participation  is  certainly  required. 
Users  know  in  general  what  they  need,  and  they  have  to  make  the 
subsystems  into  capabilities.  The  more  they  know  about  what  they 
are  getting,  the  better.  In  the  rush  to  apply  this  wisdom  to  the  acqui¬ 
sition  process,  some  misjudgments  have  been  made. 

A  true  development  activity  cannot  be  accomplished  by  the  using 
agency,  since  it  does  not  have  the  required  resources.  On  the  other 
hand,  it  is  important  that  the  agency  be  involved  to  help  shape  the 
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resulting  capability  and  to  help  in  the  transition  from  development  to 
operational  use.  Development  must  be  done  where  the  development 
talent  exists,  not  necessarily  at  a  user  headquarters.  Ideally,  the  user 
will  provide  personnel  at  the  optimum  development  location  to 
achieve  the  necessary  user  participation.  Again,  ideally,  these  user 
personnel  will  move  into  the  operational  locations  as  the  newly  devel¬ 
oped  system  moves  into  field  operation.  Programs  have  failed  for  lack 
of  adequate  user  participation.  However,  they  have  also  failed  when 
they  were  attempted  at  user  ItKations  and  adequate  development 
resources  were  not  brought  to  bear  on  the  scene.  As  in  most  other 
considerations,  each  acquisition  program  must  plan  for  proper  user 
and  developer  participation.  There  are  no  fixed  rules,  except  that 
both  groups  must  be  properly  represented,  or  failure  or  disappoint¬ 
ment  with  the  resulting  capability  is  likely. 

As  the  earlier  quotation  from  Beam  makes  clear,  requirements  are 
not  chiseled  in  stone  and  handed  down  from  the  mountain.  Even 
when  approved  requirements  statements  are  available,  they  leave 
much  room  for  interpretation.  It  is  MITRE’s  task  as  systems  engineer 
to  take  those  operational  rcquiren'cnrs  and  translate  them  into  tech¬ 
nical  performance  requirements.  That  process  results  in  a  technical 
specification  that  is  used  by  the  SPO  to  hire  a  ct)ntractor  to  build  the 
required  subsystem  in  such  a  way  that  when  it  is  combined  with 
other  subsystems,  it  will  provide  the  user  command  with  the  required 
capability.  For  that  to  happen  successfully,  the  C.'orporation  must 
understand  not  only  the  requirements  associated  with  the  current 
program,  but  how  the  resulting  system  fits  into  the  overall  mission 
system  capability. 

As  part  of  the  process  of  establishing  requirements,  MITRE  has  a 
responsibility  to  state  what  is  technically  feasible  within  the  con¬ 
straints  that  may  apply  to  the  program,  such  as  available  time  and 
money.  The  Corporation  often  has  experience  relevant  to  the  capabil¬ 
ity  in  question,  and  based  on  that  experience  should  propose  achiev¬ 
able  system  requirements.  In  addition,  it  should  comment  on  what 
others  may  propose  as  system  requirements.  MITRE's  considerations 


should  include  technical  feasibility,  associated  risk,  likely  cost  and 
schedule,  impact  on  or  by  other  programs,  and  alternative  require¬ 
ments.  Although  not  often  discussed,  in  the  requirements  formulation 
stage  MITRE  can  have  a  significant  impact  on  the  capability  that  is 
eventually  achieved.  The  Corporation  needs  to  be  proactive  in  this 
stage,  not  merely  reactive. 

As  with  everything  else  in  the  acquisition  of  major  system  capabili¬ 
ties,  over  the  course  of  the  program,  there  will  be  changes  proposed  in 
the  operational  requirements.  A  change  in  threat  or  user  employment 
plans  may  occur.  Changes  may  even  be  proposed  by  MITRE.  Often, 
people  or  organizations  will  attempt  to  get  the  program  director  to 
agree  to  these  changes  without  going  through  any  formal  approval 
process.  If  the  change  is  significant — that  is,  if  it  affects  performance, 
cost,  or  schedule  in  ways  that  significantly  increase  the  risk  of  successful 
completion  of  the  program — MITRE  should  recognize  that  fact.  In  turn, 
the  Corporation  should  strongly  urge  the  system  program  director  who 
manages  the  SPO  to  defer  from  making  the  change  until  it  is  formally 
approved,  and  until  the  resources  necessary  to  accomplish  it  are  pro¬ 
vided.  MITRE  should  help  the  program  director  to  estimate  the  perfor¬ 
mance,  time,  and  cost  implications  of  the  proposed  changes.  These 
implications  should  be  made  known  to  both  the  user  community  and  to 
those  who  provide  the  funding  for  the  acquisition  program.  By  clearly 
identifying  the  implications,  this  approach  will  weed  out  those  changes 
in  which  the  uset  command  is  not  seriously  interested.  It  will  also  elimi¬ 
nate  those  for  which  adequate  funding  and  time  have  not  been  provided 
to  the  development  command. 

Occasionally,  a  user  command  will  desire  a  new  capability  but 
reject  the  approach  suggested  for  achieving  it.  For  many  years,  the 
predecessors  of  the  Federal  Aviation  Agency  performed  air  traffic 
control  using  raw,  or  video,  radar  information.  When  capacity  and 
accuracy  requirements  for  air  traffic  control  dictated  the  application 
of  computers,  it  was  necessary  to  digitize  the  video  information  so  it 
could  be  processed  by  computers.  However,  the  controller  agency 
was  comfortable  with  the  use  of  video  and  concerned  that  the 


digitizing  process  might  eliminate  the  display  ot  certain  aircraft,  even 
though  video  was  present.  As  a  result,  for  several  years,  both  the 
video  and  processed  radar  information  had  to  be  displayed  to  air 
traffic  controllers.  The  processed  data  was  provided  to  achieve  the 
increased  system  performance  that  the  air  traffic  situation  demanded 
and  that  the  government  required.  The  display  of  video  was  main¬ 
tained  in  the  new  system  until  controllers  were  convinced,  by  the 
system  performance,  of  the  validity  of  processed  radar  c  ;.i  as  a  basis 
for  a  safe  air  traffic  control  system.  Such  problems  are  not  unique  to 
air  traffic  control.  The  use  of  digital  data  communications  in  place  of 
voice  has  been  resisted  by  some  factions  of  the  military  on  similar 
grounds.  As  was  done  in  the  air  traffic  control  case,  the  design  of  the 
new  capability  must  accommodate  these  very  real  considerations,  and 
MITRE  must  be  especially  sensitive  to  them. 

Other  unusual  requirements  problems  may  arise  at  any  time  in  a 
program.  In  Southeast  Asia,  there  was  a  system  that  controlled  rescue 
flights  over  North  Vietnam  in  attempts  to  retrieve  downed  airmen. 
When  the  losses  on  those  flights  rose  to  unacceptable  levels,  the 
Commanding  General  of  the  7th  Air  Force  instituted  a  requirement 
that  he  personally  approve  each  flight.  These  were  very  important 
missions  and  also  among  the  most  dangerous.  The  Commanding 
General  wished  to  personally  judge  the  likelihood  of  success  in  each 
case  and  to  weigh  that  against  the  potential  for  further  losses.  To 
provide  him  with  detailed  information  at  his  headquarters  location, 
the  system  involved  was  quickly  modified. 

The  last  example  is  reminiscent  of  a  general  problem  that  always 
exists  in  establishing  requirements  for  a  new  system.  What  decisions 
will  be  made  at  each  operating  level  within  the  using  command?  How 
much  of  all  the  information  available  within  the  system  will  be  pro¬ 
vided  to  each  command  level?  If  all  information  is  available  at  all 
levels,  the  higher  levels  may  be  overwhelmed  by  too  much  data.  Or, 
the  lower  levels  may  be  uneasy  that  higher  levels,  especially  those 
outside  their  service  command,  may  usurp  their  functions.  If  too  little 
data  is  provided  to  higher  levels,  important  decisions  may  be  made 


erroneously.  This  is  another  area  in  which  MlTRE’s  experience  on 
earlier  versions  of  the  system,  or  on  other  related  systems,  may  be 
quite  helpful  in  recommending  a  sensible  distribution  of  functions 
and  information.  Perhaps  the  system  design  needs  to  be  flexible, 
allowing  for  alternative  operations  as  the  situation  may  dictate. 

In  establishing  system  requirements,  the  user  command  must  also 
describe  any  differences  between  those  of  peace  and  war.  In  peace, 
the  cost  of  operation  is  always  a  major  consideration,  but  less  so  in 
war,  when  effectiveness  transcends  other  factors.  A  system  may  nor 
have  a  day-to-day  function  except  during  war,  yet  it  must  be  avail¬ 
able  on  short  notice,  personnel  must  be  trained,  and  it  must  operate 
reliably.  One  thought  that  has  been  impressed  on  MITRE  by  military 
commanders  over  the  years  is  that  one  cannot  expect  to  use  a  system 
in  a  certain  way  during  peace,  and  then  be  successful  at  operating  it 
differently  during  war.  Capacities  may  be  different  in  peace,  or  some 
additional  checks  required  before  taking  action,  but  fundamental 
system  operation  must  be  the  same. 

What  does  this  all  mean  to  MITRE  as  systems  engineer  for  the 
proposed  acquisition  program?  One  of  the  main  points  of  this  book  is 
that  to  be  effective  in  the  systems  engineering  role,  MITRE  personnel 
must  have  a  genuine  understanding  of,  and  to  the  extent  possible,  a 
real  experience  with  the  capability  that  the  system  is  to  provide. 

Of  all  the  qualities  and  capabilities  that  MITRE  brings  to  the 
acquisition  of  command  and  control  systems,  this  appreciation  for  the 
capabilities  desired  by  the  government  is  uniquely  MlTRE’s.  What  it 
means  for  MITRE  to  understand  the  desired  capability,  how  that 
understanding  is  acquired,  and  how  it  is  applied  in  any  particular 
acquisition  program  are  detailed  in  Chapter  4. 

Complicated  though  the  process  may  be,  the  government  has  many 
mechanisms  to  study  the  effectiveness  of  the  various  military  capabilities 
and  to  evaluate  potential  improvements.  Existing  systems  are  subject  to 
daily  use.  Special  operational  exercises  and  tests  are  conducted  to  help 
evaluate  system  performance  in  ways  not  stressed  by  day-to-day  opera¬ 
tion.  New  systems  go  through  development  testing  and  readiness  testing 
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bel’ore  becoming  operational.  All  of  these  activities  help  to  establish  the 
level  ot  performance  of  the  existing  capabilities  and  provide  indications 
of  where  improvements  may  be  necessary.  Beyond  that,  the  Air  Force 
and  the  other  services  fund  mission  system  planning  activities  to  stiid> 
new  potential  system  capabilities,  including  considerations  of  technology, 
operations,  and  costs.  On  (Kcasion,  the  Air  Force  establishes  and  funds 
conceptual  planning  studies  at  .\11TRE.  For  major  upgrades  to  existing 
systems  on  which  MITRE  has  the  systems  engineering  role,  it  is  quite 
common  for  MITRE  to  do  the  conceptual  planning  for  the  upgrade  as 
part  of  the  funded  systems  engineering  work. 

In  addition  to  the  resources  available  within  the  operating  and 
development  communities,  the  government  employs  a  variety  of  other 
groups  to  help  evaluate  current  performance  and  to  study  potential 
new  systems.  These  include,  for  example,  the  Defense  Science  Board 
and  the  Air  Force  Scientific  Advisory  Board.  If  appropriate,  help  may 
be  enlisted  from  other  groups,  such  as  the  National  Academy  of 
Sciences,  which  are  less  directly  involved  in  DOD  matters.  Special  ad 
hoc  groups  may  be  set  up  to  study  important  problems.  More  than 
one  of  these  groups  may  examine  a  new  technology  for  potential 
applications  or  a  critical  operational  need  for  possible  technical 
solutions.  In  doing  so,  they  interact  extensively  with  the  development 
and  operational  communities. 

The  MITRE  technical  staff  has  been  privileged  for  many  years  to 
participate  in  a  wide  range  of  these  activities  both  as  a  part  of  the 
development  community  and  as  contributors  to  tiie  special  govern¬ 
ment  studies  performed  by  groups  such  as  those  just  mentioned.  In 
the  early  1960s,  MITRE  staff  helped  the  Air  Force  evaluate  alterna¬ 
tives  for  survivable  continental  air  defense  systems  that  led  to  the 
acquisition  of  the  Backup  Interceptor  Control  (BUIC)  air  defense 
system.  The  Corporation  was  an  important  part  of  the  studies  that 
led  to  a  series  of  acquisition  programs  for  upgrading  the  ability  of  Air 
Force  Ci  systems  to  support  tactical  mission  operations  anywhere  in 
the  world.  MITRE  has  performed  similar  work  in  strategic  mission 
areas  such  as  missile  warning  and  space  defense.  In  some  cases,  the 


government  has  funded  the  Corporation  to  perform  the  separate 
studies;  in  other  instances,  MITRE  staff  members  have  participated  as 
members  of  Air  Force  or  other  study  groups.  MITRE  and  industry 
studies,  together  with  Air  Force-funded  advanced  development  pro¬ 
grams  conducted  by  the  Corporation,  led  to  the  establishment  of  the 
Joint  Tactical  Information  Distribution  (JTIDS)  program.  Similar 
MITRE  work  helped  the  Air  Force  achieve  the  Flave  Quick  voice 
radio  antijam  capability. 

MITRE  staff  members  have  participated  in  external  studies  con¬ 
ducted  by  the  Defense  Science  Board  and  many  other  special  study 
groups.  The  subjects  to  which  MITRE  personnel  have  contributed 
range  over  areas  such  as  ground  target  surveillance  and  attack,  C^I 
systems  countermeasures  and  counter-countermeasures,  C’l  systems 
for  next-generation  ballistic  missiles,  survivable  communications, 
supportability  of  U.S.  mission  system  capabilities,  and  the  Strategic 
Defense  Initiative  (SDI).  The  combination  of  in-depth  technical 
knowledge  and  understanding  of  the  operational  missions  found  in 
the  Corporation’s  technical  staff  has  resulted  in  it  being  requested  to 
participate  in  these  studies.  That  combination  has  also  helped  the  staff 
to  make  significant  contributions  to  the  results.  In  return,  the  MITRE 
staff  has  learned  much  from  the  experience  and  has  established 
important  relationships  in  government,  academia,  and  industry.  The 
knowledge  and  rapport  gained  are  important  to  the  effectiveness  of 
the  Corporation’s  systems  engineering  work. 

As  one  might  infer  from  the  material  just  presented,  conceptual 
planning  for  a  new  system,  or  for  a  major  addition  to  an  existing 
system,  is  not  always  a  bounded,  well-disciplined  process.  Many 
voices  have  to  be  heard.  Needs  may  spring  from  the  user  community. 
System  ideas  may  be  proposed  by  industry,  MITRE,  academia,  or  any 
of  the  many  study  groups.  Industry  or  academia  may  offer  technology 
without  knowing  where  it  might  best  be  applied.  Criticism  of  current 
capabilities  may  come  from  any  quarter  and  may  be  legitimate  or 
biased,  knowledgeable  or  emotional.  In  this  sort  of  environment, 
MITRE  must  be  mindful  of  its  role — to  help  the  government  achieve 


required  eupabilitie'.  on  reasonable  schedules  and  at  reasonable  costs. 
Objective,  inlormed  assessment  »)t  all  rele\ant  tactor^  must  be  the 
basis  lor  MITRH's  recottimendations  on  hou  best  to  satish  the 
government's  needs.  .Ml  I'RF  must  be  prepared  to  detend  its  recom¬ 
mendations  in  both  government  and  other  lorums. 

With  sonte  knowledge  ot  what  may  be  desirable  and  what  may  be 
possible  to  do,  .MITRF.  hypothesizes  broad  alternative  system  designs. 
Fach  design  is  studied  tor  likely  system  pertormance  and  assessment 
ot  inherent  risks.  System  and  siipptirt  exists  and  schedules  are  esti¬ 
mated.  The  potential  threat  and  the  environment  in  which  the  system 
must  be  approved,  developed,  and  operated  are  factored  into  the 
concept,  schedules,  and  costs.  .MITRF's  work  is  subject  to  scrutiny  by 
others  within  the  acquisition  community,  as  well  as  by  various  study 
groups,  and  otten  by  industry.  The  risks  are  evaluated  b\  the  (Corpo¬ 
ration  through  further  analysis  or  experimentation. 

As  results  are  determined  and  comments  received  from  other  par¬ 
ties,  the  Corporation  refines  the  alternative  system  approaches  and 
adjusts  the  estimates  of  performance,  risk,  schedule,  and  cost.  MITRF. 
provides  the  results  to  the  government,  along  with  recommendations 
on  how  to  proceed.  MITRF.  is  prepared  to  participate  in  the  delibera¬ 
tions  at  that  time  in  any  way  deemed  helpful  by  the  government.  The 
Corporation  is  eager  to  be  an  active  participant  because  it  feels  it  has 
something  substantive  to  contribute  to  the  decision-making  process, 
and  because  the  insight  provided  by  direct  participation  is  especially 
helpful  in  understanding  the  government's  de  i'^xins  and  in  carrying 
them  out  when  MITRE  is  assigned  the  systems  engineering  role. 
MITRF’s  active  participation  is  important  both  to  the  C.'orporation 
and  to  the  government.  With  care  on  both  sides,  it  can  be  accom¬ 
plished  without  any  real  or  apparent  compromise  or  abrogation  of 
the  government's  ultimate  responsibility  for  making  the  decisions 
about  what  will  or  will  not  he  acquired  and  how. 

In  many  cases,  there  are  substantial  predetermined  constraints  that 
must  be  respected  in  any  concept  planning  or  concept  definition 
activity.  In  one  example,  MITRE  was  asked  to  define  alternative 


systems  for  improved  survivability  of  the  continental  air  defense 
system,  but  the  total  cost  could  not  exceed  $100  million.  In  another 
instance,  an  interface  capability  was  required  between  two  tactical 
systems,  but  it  had  to  be  operational  in  less  than  a  year.  Meeting  the 
requirements  was  possible  in  both  of  these  cases.  However,  if  too 
many  of  the  attributes  of  performance,  schedule,  or  cost  are  fixed  a 
priori,  the  possible  system  solutions  may  be  seriously  constrained.  It 
is  conceivable  that  meeting  all  the  constraints  is  undesirable  or  even 
l.npossible.  In  such  cases,  MITRE  must  be  prepared  to  recommend 
how  best  to  compromise  among  the  factors  driving  what  can  be  done, 
or  even  to  recommend  that  the  program  should  not  proceed  under 
the  given  circumstances. 

Whenever  a  conceptual  study  of  a  potential  new  system  or  capabil¬ 
ity  is  undertaken,  MITRE  must  have  discussions  with  all  the  key 
players.  This  includes  those  involved  in  the  study,  the  using  com¬ 
mand,  others  who  may  participate  in  approving  the  system  for  acqui¬ 
sition  or  in  developing  it,  those  whose  expertise  may  contribute  to  a 
successful  program,  and  any  factions  that  are  likely  to  dissent.  Only 
in  that  way  will  MITRE  obtain  a  true  picture  of  the  government’s 
needs  and  constraints,  what  may  or  may  not  be  possible,  and  the 
environment  surrounding  the  potential  system.  As  has  been  observed, 
two  of  the  attitudes  that  distinguish  MITRE’s  approach  to  systems 
engineering  are  in-depth  understanding  of  the  operational  require¬ 
ments  and  familiarity  with  the  total  environment  within  which  system 
acquisition  and  operation  take  place.  The  Corporation’s  recommen¬ 
dations  are  only  as  good  as  its  staff’s  knowledge  of  both  these  factors 
and  the  technologies  involved  in  accommodating  them. 

The  key  questions  that  MITRE  should  help  to  answer  in  the  con¬ 
cept  definition  phase  include  the  following:  What  is  the  proposed 
operational  capability?  What  are  some  of  the  alternate  levels  of  capa¬ 
bility?  What  are  the  different  approaches  to  achieving  them?  What  are 
the  schedule  and  cost  implications  of  each?  What  are  the  associated 
risks?  What  actions  should  be  initiated  to  reduce  the  risks  to  accept¬ 
able  levels?  What  areas  need  special  attention  during  acquisition? 


Who  has  strong  opinions?  How  may  they  affect  the  program?  What 
should  be  done  to  mitigate  their  concerns  or  correct  any  of  their 
misconceptions?  What  is  happening  in  other  related  programs  that 
may  impinge  on  the  capability?  How  will  developing  technology  affect 
future  capabilities  in  this  area?  Is  it  timely  to  proceed  now  or  would  it 
be  prudent  to  delay  the  program  for  some  time? 

In  summary,  MITRE  can  assume  a  very  important  role  in  helping 
the  government  to  analyze  alternatives  and  establish  system  require¬ 
ments.  The  Corporation  has  much  relevant  experience,  understand¬ 
ing,  and  information — about  both  the  operational  capabilities  and  the 
technologies — that  can  be  helpful  to  those  who  are  responsible  for 
deciding  the  requirements  for  a  particular  system.  The  Corporation  is 
responsible  for  providing  that  background  throughout  the  life  of  an 
acquisition  program,  especially  in  the  phase  when  the  initial  require¬ 
ments  are  being  established. 

Activities  Preceding  Industry  SoTicitntien 

When  the  government  decides  to  acquire  a  new  capability,  it  pub¬ 
lishes  a  program  management  directive  (PMD)  outlining  the  nature  of 
the  capability  to  be  acquired  and  describing  the  constraints  under 
which  it  is  to  be  acquired.  For  example,  the  PMD  identifies  the  avail¬ 
able  funds.  Quite  often,  the  capability  reflected  in  the  PMD  varies 
from  those  proposed  in  the  concept  design  alternatives.  In  that  case, 
further  study  and  analysis  must  be  done  by  MITRE  to  help  determine 
how  best  to  carry  out  the  program  direction.  Clarification  and  modi¬ 
fication  of  program  direction  may  be  required  to  ensure  a  good 
foundation  for  the  program.  MITRE  should  work  as  part  of  the  SPO 
team  to  obtain  them. 

Sometimes  the  PMD  is  assigned  to  the  system  program  director  of 
an  existing  SPO.  Where  there  is  no  appropriate  SPO,  a  new  one  is 
created.  In  either  case,  the  program  director  hires  MITRE  as  systems 
engineer  when  the  Corporation  is  the  best  source  of  that  support. 

MITRE  makes  one  of  its  most  important  contributions  in  the 
period  between  the  government’s  decision  to  acquire  a  system  and  the 


time  that  industry  proposals  to  build  it  are  solicited.  In  that  period, 
the  Corporation  prepares  the  system  specification  that  will  be  used  as 
the  basis  for  soliciting  the  industry  proposals.  The  specification  will 
also  be  used  in  subsequent  programs  phases  as  a  benchmark  for 
assessing  the  progress  of  the  program.  MITRE  also  participates  in 
other  pre-contract  solicitation  activities,  including  helping  to  prepare 
the  operational  employment  plan  and  the  acquisition  strategy. 

The  OperotioiMl  Employiiient  Plan 

When  the  dust  has  settled  on  all  the  “what  ifs"  studied  in  the 
concept  definition  phase,  it  is  a  worthwhile  exercise  to  generate  a 
plan  to  describe  how  the  user  intends  to  employ  the  system  that  has 
been  approved  and  funded  for  acquisition.  Historically,  such  a  plan 
was  often  generated  on  ESC-developed  systems.  It  is  strongly  recom¬ 
mended  that  such  a  plan  be  generated  for  all  new  systems.  In  the 
past,  the  plan  was  referred  to  as  an  opetational  employment  plan,  or 
OEP.  The  utility  of  such  a  document  is  becoming  more  widely  recog¬ 
nized.  For  example,  the  Air  Force  Ballistic  Missile  Organization  has 
published  a  draft  preparation  guide  for  a  Baseline  Concept/System 
Description  document.^*  As  described  in  the  draft  guide,  this  docu¬ 
ment  is  similar  to,  but  somewhat  more  narrow  in  scope  than  the 
OEPs  produced  on  ESC  programs. 

The  OEP  may  be  written  by  the  using  command  or  it  may  be  a 
product  of  a  combined  user-developer  group.  Very  often,  MITRE’s 
understanding  of  both  the  operational  needs  and  the  associated  tech¬ 
nologies  permit  the  Corporation  to  make  significant  contributions  to 
the  OEP.  In  many  ways,  by  publishing  the  OEP,  the  using  command 
is  saying,  “If  you  build  me  a  system  that  meets  the  requirements  of 
this  plan,  then  you  will  have  satisfied  my  requirements.”  Such  a  state¬ 
ment  is  beneficial  for  understanding  among  parties.  It  is  a  statement 
that  will  either  survive  the  key  personnel  changes  that  take  place 
during  a  program  or  provide  a  basis  for  equitably  negotiating  change. 

Baseline  Concept/System  Description  Document  Preparation  Guide,  Draft, 
Ballistic  Missile  Organization,  Norton  Air  Force  Base,  CA,  May  1992. 


In  the  OEP,  the  using  command  interprets,  in  operational  terms, 
what  it  believes  has  been  approved  for  acquisition.  The  document 
states  how  the  command  intends  to  employ  and  support  the  new 
capability,  and  how  it  will  interoperate  with  the  other  existing  and 
planned  portions  of  their  mission  capability.  It  is  an  opportunity  for  a 
close  dialog  between  user  and  developer  to  resolve  any  last  minute 
confusion  over  what  is  desired  and  what  will  be  delivered,  before 
industry  is  put  on  contract  to  build  it.  The  OEP  is  also  a  governor  on 
future  changes  that  may  be  required  by  the  user  or  on  future  disputes 
about  what  was  done  versus  what  was  supposed  to  have  been  done. 
Future  needs  may  be  justifiably  different.  Good  management  requires 
that  the  foundation  for  the  program  be  described  and  agreed  to  by 
the  user  and  developer.  Changes  can  then  be  accommodated  by 
changing  the  agreement.  This  approach  provides  a  businesslike  basis 
for  renegotiating  the  resources  available  to  the  developer  for  achiev¬ 
ing  the  revised  capability. 

Since  the  OEP  is  normally  published  by  the  using  command,  it  is  a 
command-level  statement  of  requirements,  rather  than  merely  a 
statement  by  one  or  more  command  personnel.  As  such,  it  stands  the 
test  of  time  more  adequately.  That  is  especially  important  in  future 
negotiations  of  program  change.  If  events  are  such  that  changes  in 
operational  requirements  dictate  modifications  to  the  system,  the 
OEP  provides  a  basis  for  negotiating  them.  The  OEP  agreement  also 
helps  to  define  the  test  programs  that  will  be  conducted  as  part  of  the 
acquisition  program  to  demonstrate  that  the  system  works  as  planned 
and  that  resulting  capability  is  operationally  acceptable.  Since  expec¬ 
tations  and  people  change  as  time  passes,  the  OEP  improves  the 
ability  of  the  participants  to  manage  more  equitably  the  impact  of 
change  on  the  required  test  programs. 

By  participating  actively  in  the  OEP  process,  MITRE  gains  insight 
into  what  the  user  wants  in  the  capability.  In  turn,  that  insight  is 
valuable  to  the  Corporation’s  work  in  preparing  the  system  specifica¬ 
tion  used  in  soliciting  industry.  MITRE  can  also  help  improve  the 
user’s  understanding  of  the  capability  to  be  provided  and  the  risks 


associated  with  trying  to  do  so.  The  key  questions  associated  with  the 
OEP  phase  are:  Is  there  a  good  understanding  between  the  user  and 
the  developer  on  what  is  required  and  how  it  will  be  provided?  Does 
it  reflect  the  program  direction?  Is  the  understanding  sufficient  to 
proceed  with  the  program? 

AcqwsitiM  Strategy 

When  a  decision  is  made  to  acquire  a  new  capability  and  a  PMD  is 
issued  to  initiate  the  program,  a  system  program  director  is  also 
appointed  to  manage  the  acquisition.  Some  years  ago,  the  phrase 
“acquisition  strategy”  was  invented  to  cover  a  myriad  of  decisions 
that  the  program  director  must  make  in  establishing  the  acquisition 
approach  to  be  followed.  The  program  director  is  not  a  completely 
free  agent  in  this  process.  Many  of  his  or  her  decisions  are  subject  to 
review  by  higher  headquarters  and  to  coordination  with  other  partici¬ 
pants,  such  as  the  using  command  and  the  various  commands  respon¬ 
sible  for  trainmg,  construction,  and  logistics.  Sometimes,  direction 
given  to  the  program  director  specifies  in  part  how  the  acquisition 
will  be  managed. 

The  acquisition  strategy  for  a  program  may  include  many  different 
factors.  Will  there  be  a  prime  or  system  contractor?  If  not,  how  will 
the  capability  be  broken  into  pieces  that  can  be  contracted  for  sepa¬ 
rately?  What  form  will  the  contract  take,  fixed  price  or  other?  Will 
contract  incentives  be  used?  Will  the  capability  be  provided  in  incre¬ 
mental  steps?  Will  there  be  a  study  phase  before  actual  development? 
How  will  the  contractor  be  selected? 

In  discussing  acquisition  strategy  here,  two  points  are  emphasized. 

The  decisions  made  in  this  phase  are  critical  factors  in  the  success  or 
failure  of  the  program.  Experience  clearly  shows  that  good  decisions 
facilitate  a  successful  prt^am  and  that  poor  ones  are  a  prescription  for 
trouble  and,  in  some  cases,  failure  in  the  acquisition  program.  The  sec¬ 
ond  point  is,  as  systems  engineer,  MITRE  must  take  the  initiative  to 
actively  participate  in  the  decisions  concerning  acquisition  strategy.  The 
Corporation  has  the  benefit  of  experience  with  the  approaches  that  did 


or  did  not  work  on  similar  past  programs.  In  addition,  many  of  the 
questions  to  be  answered  in  establishing  the  acquisition  strategy  have 
significant  technical  components  that  must  be  considered.  For  example, 
in  this  phase  of  a  program,  MITRE  has  a  greater  understanding  of  the 
amount  of  development  that  will  be  required  than  any  other  program 
participant.  This  knowledge  should  be  brought  to  bear  on  decisions,  such 
as  the  need  for  system  studies  or  development  models  and  even  the 
contrart  form  to  be  used.  As  systems  engineer,  MITRE  must  take  a  very 
active  approach  to  understanding  the  key  issues  on  the  program  and  to 
providing  appropriate  recommendations  to  the  program  director  on  the 
acquisition  strategy  to  be  employed. 

Certain  management  approaches  have  tended  to  be  in  vogue  at 
different  times.  For  example,  there  have  been  times  when  the  pre¬ 
ferred  acquisition  approach  was  to  hire  a  prime  or  a  system  contrac¬ 
tor  and  to  make  that  contractor  responsible  for  providing  the  total 
system.  At  other  rimes,  conventional  wisdom  dictated  that  the  gov¬ 
ernment  would  achieve  more  for  its  money  if,  for  example,  it  hired 
the  best  hardware  contractor  and  then  separately  contracted  with  the 
best  software  firm.  A  third  contractor  might  be  hired  to  integrate  the 
efforts  of  the  first  two,  or  the  government  might  assume  that  role 
with  assistance  from  MITRE.  Selecting  the  approach  to  be  used  is  an 
important  choice,  and  each  alternative  has  associated  pros  and  cons. 
Often,  the  choice  has  extremely  important  technical  ramifications, 
and  the  Corporation’s  expertise  can  help  highlight  the  risks  and 
advantages  of  the  available  alternatives.  For  example,  in  a  case  where 
the  hardware  and  the  operational  software  are  to  be  procured  sepa¬ 
rately,  one  must  take  great  care  in  deciding  which  contractor  will 
provide  the  support  software,  such  as  the  operating  system.  That  is  a 
difficult  decision,  one  that  can  make  or  break  the  program,  and  one 
that  is  in  large  measure  driven  by  technical  factors  on  which  MITRE 
has  considerable  expertise  and  experience. 

As  in  all  matters  associated  with  the  acquisition  of  large  and  com¬ 
plicated  C^I  systems,  there  is  no  universally  optimum  prescription. 
Each  instance  has  to  be  evaluated  on  its  own  merits.  Some  failures  in 


acquisition  programs  can  be  traced  directly  to  acquisition  strategies 
inappropriate  for  the  program  at  hand.  Having  a  single  system  con¬ 
tractor  significantly  reduces  the  negotiations  the  SPO  must  conduct 
with  industry.  A  single  contractor  may  give  the  SPO  more  flexibility 
in  managing  tradeoffs  between  the  various  subsystems  that  constitute 
the  capability.  With  a  cooperative  system  contractor  there  is  increased 
flexibility;  with  a  reluctant  one,  there  is  less. 

The  need  to  accommodate  change  is  omnipresent  in  the  acquisition 
of  large  C^I  systems.  A  cooperative,  skilled  system  contractor  im¬ 
proves  the  chances  for  accommodating  change  with  least  possible 
overall  impact  on  the  program.  An  uncooperative  or  unskilled  system 
contractor  can  make  change  a  very  difficult  and  costly  process.  One 
might  be  concerned  that  when  there  is  a  system  contractor,  the  SPO 
has  no  alternative  but  to  make  any  necessary  changes  through  that 
contractor.  Knowing  that,  the  contractor  may  suggest  high  costs  for 
any  change.  On  the  other  hand,  the  government  has  perfectly  ad¬ 
equate  mechanisms  for  negotiating  “should  cost”  in  a  reasonable 
way.  The  system  contractor  may  make  changes  internal  to  the  system 
that  the  government  does  not  like  while  still  claiming  to  deliver  the 
overall  capability.  For  example,  the  system  contractor  may  decide  for 
business  reasons  to  take  work  away  from  an  important,  skilled  sub¬ 
contractor  and  move  it  into  his  own  organization. 

In  all  these  cases,  MITRE  can  make  valuable  assessments  of  the 
technical  ramifications  of  the  contractor’s  actions.  The  Corporation  is 
able  to  make  estimates  of  the  work  involved  in  a  change  and  there¬ 
fore  can  help  calibrate  the  cost  and  schedule  impacts  of  the  changes. 
mitre’s  work  on  technical,  cost,  and  schedule  impacts  helps  provide 
the  SPO  with  substantive  information  as  a  basis  for  government 
evaluation  of  the  contractor  actions. 

When  there  is  no  system  contractor,  the  government  must  either  do 
the  work  required  to  integrate  the  pieces  itself  or  hire  another  contrac¬ 
tor  to  do  so.  The  government  is  very  short  of  the  professional  resources 
necessary  to  perform  integration  functions.  Hiring  an  integration  con¬ 
tractor  introduces  an  industrial  component  that  has  no  control  over  the 


Whether  there  is  o  system 
controctor  or  not,  the  progrom 
director  ond  MITRE  must 
appreciate  that  if  the  acquisition 
program  fails  in  some 
important  way,  they  will  be 
held  responsible. 


contractors  building  the  subsystems,  requires  bringing  that  contractor 
up  to  speed  on  the  capabilities  required  and  the  technology  involved, 
and  requires  the  government  to  be  prepared  to  referee  disputes  that  will 
inevitably  take  place  between  the  various  industrial  corporations.  Some 
critics  claim  the  aciivity  is  an  unnecessary  overhead  placed  on  top  ot  the 
contractors  providing  the  subsystems.  When  MITRE  was  first  formed, 
the  number  of  systems  hieing  procured  was  small  and  the  Corporation 
was  able  to  assist  the  government  by  performing  in  the  system  integra¬ 
tor  role.  Today,  MITRE  continues  to  provide  significant  support  in  this 
area,  but  limitations  on  technical  staff  availability  preclude  the  possibil¬ 
ity  of  the  Corporation  assuming  total  integration  responsibility  in  all 
but  the  most  special  circumstances. 

Although  some  people  may  disagree,  it  seems  clear  that  when  in 
doubt,  the  best  approach  is  to  hire  a  system  contractor.  Care  must  be 
taken  to  select  one  that  has  demonstrated  competence  in  the  type  of 
system  being  acquired  and  has  the  necessary  resources  available  for 
assignment  to  the  program.  The  contractual  arrangements  with  the 
system  contractor  must  be  such  that  the  SPO  remains  in  charge  of  its 
destiny,  retains  the  necessary  visibility  into  the  system  contractor’s 
activities,  and  is  not  completely  at  the  mercy  of  the  contractor  man¬ 
agement  when  changes  are  necessary.  It  is  particularly  important  that 
there  is  adequate  visibility  into  the  activities  of  the  subcontractors  so 
that  MITRE  has  the  information  required  to  provide  technical  assess¬ 
ments  of  program  status  and  to  help  in  evaluating  changes  initiated 
by  the  contractor  or  the  government. 

Whether  there  is  a  system  contractor  or  not,  the  program  director 
and  MITRE  must  appreciate  that  if  the  acquisition  program  fails  in 
some  important  way,  they  will  be  held  responsible.  If  the  program 
does  not  provide  the  required  operational  capability,  both  MITRE 
and  the  government,  as  well  as  the  contractor,  have  failed.  When 
there  is  a  system  contractor,  the  Corporation  must  continue  to  do 
everything  it  normally  does  to  help  the  government  acquire  the  re¬ 
quired  capability.  When  there  is  not,  MITRE  must  also  be  prepared 
to  help  the  government  provide  the  necessary  integration  functions. 


Another  aspect  of  acquisition  strategy  involves  creating  as  much 
industrial  competition  as  possible.  The  more  qualified  bidders  there 
are,  the  more  likely  that  innovative  approaches  will  be  proposed  and 
that  systems  costs  will  be  as  low  as  practical.  Competitive  study 
contracts  are  sometimes  employed  as  part  of  the  acquisition  strategy. 
Such  studies  provide  the  government  information  on  how  a  capability 
might  be  achieved,  possible  levels  of  performance,  risk  areas,  and 
associated  costs  and  schedules.  They  help  identify  strengths  and 
weaknesses  among  the  participating  contractors.  The  contractors 
learn  more  about  the  capability  required  and  how  it  may  be  provided. 
Often,  the  alternatives  studied  involve  competing  state-of-the-art 
technologies.  Direct  participation  by  MITRE  helps  guide  the  con¬ 
tractors  to  examine  the  most  critical  design  and  performance  ques¬ 
tions.  The  Corporation’s  expertise  in  modern  technologies  is  available 
to  help  evaluate  the  results  and  quantify  the  probable  cost  and  sched¬ 
ule  impacts  of  various  approaches  suggested  by  the  contractors.  The 
Corporation  also  can  help  apply  the  results  to  the  subsequent  acqui¬ 
sition  program  by  including  appropriate  information  in  the  MITRE- 
prepared  system  specification.  The  information  gained  is  also  useful 
in  the  technical  evaluation  of  the  contractor  system  acquisition  pro¬ 
posals,  and  in  the  evaluation  of  contractor  progress  as  the  program 
proceeds. 

On  the  other  hand,  if  improperly  used,  study  contracts  can  be  a 
waste  of  the  government’s  time  and  money.  To  the  extent  possible, 
the  study  contractors  should  be  corporations  that  are  capable  of 
participating  in  the  actual  development  of  the  system.  If  possible, 
contractors  solicited  for  developing  the  system  should  be  limited  to 
those  who  executed  the  study  contracts.  At  a  minimum,  the  results 
of  the  studies  should  be  applied  to  the  acquisition  program.  As 
discussed  in  some  detail  in  the  section  on  system  design,  there  is 
often  great  reluctance  about  giving  design  direction  to  an  acquisi¬ 
tion  contractor.  This  reluctance  should  not  keep  the  government 
from  making  maximum  use  of  the  study  results  in  the  acquisition 
program. 


In  almost  every  acquisition  program,  some  portion  of  the  capability 
will  be  provided  by  industry  and  some  portion  by  the  government.  Even 
when  there  is  a  system  contractor  responsible  for  delivering  a  total  sys¬ 
tem  capability,  one  finds  that  some  piece  of  the  necessary  test  equipment 
is  government-furnished  or  some  existing  operational  equipment  must  be 
provided  to  the  contractor  for  integration  into  the  total  system.  Or 
perhaps  the  government  is  responsible  for  providing  the  facilities  in 
which  the  system  will  be  installed.  Whatever  the  specifics,  government- 
furnished  equipment  (GEE)  or  facilities  are  an  important  consideration  in 
the  formulation  of  the  acquisition  strategy.  MITRE  must  help  the  SPO 
identify  both  what  is  required  and  who  in  the  government  will  provide  it. 
This  activity  has  a  significant  technical  component  that  the  Corporation 
can  help  to  provide.  For  interfacing  systems,  what  are  the  electrical 
characteristics  that  must  be  matched?  What  message  standards  will  be 
employed?  What  changes  may  be  taking  place  in  either  system  that  might 
affect  the  interface?  The  Corporation’s  experience  with  a  wide  variety  of 
C^I  systems  can  be  very  helpful  in  identifying  the  questions  that  must  be 
answered  and  in  providing  answers  that  can  be  used  by  the  contractor 
and  by  the  government  to  ensure  a  successful  operating  interface.  The 
availability  of  electrical  power,  air  conditioning,  and  long  range  commu¬ 
nications  are  other  areas  that  often  require  MITRE  technical  contribu¬ 
tions  to  ensure  that  the  new  system  will  operate  successfully  when 
delivered  by  the  contractor. 

Negotiations  to  provide  for  GEE  or  facilities  must  be  completed 
and  subsequently  monitored  for  timely  implementation.  One  must 
also  be  concerned  about  the  characteristics  and  the  operating  condi¬ 
tion  of  the  GEE  to  be  turned  over  to  the  contractor.  The  contract 
must  include  that  information  and  provide  for  demonstrating  that  the 
equipment  is  indeed  in  the  prescribed  condition.  Failure  to  meet  the 
government’s  commitments  with  respect  to  GEE  may  cause  serious 
delays  in  the  contractor’s  work  and  attendant  cost  increases  to  the 
government. 

As  noted  earlier  in  this  book,  C^I  systems  evolve  over  time  in 
response  to  changes  in  requirements  and  in  available  technology. 


New  capabilities  are  defined,  acquired,  and  implemented  as  the  need 
arises.  However,  there  is  another  basis  for  evolutionary  development 
that  is  more  predictable;  it  is  sometimes  referred  to  as  preplanned 
product  improvement.  This  is  not  a  new  notion,  despite  what  some 
may  think.  When  the  SAGE  air  defense  system  was  conceived  in  the 
1950s,  it  was  recognized  that  new  «''nsor  and  weapon  system  devel¬ 
opments  were  in  progress  and  that  they  would  have  to  be  incorpo¬ 
rated  into  the  SAGE  system.  It  was  also  recognized  that  the  threat 
was  evolving  in  ways  that  would  provide  much  faster  enemy  bombers 
than  those  that  existed  at  the  time  SAGE  was  first  built.  The  acquisi¬ 
tion  approach  adopted  was  to  build  the  system  in  models,  with  model 
2  replacing  model  1,  model  3  replacing  model  2,  etc.  That  way,  an 
early  capability  was  achieved,  advanced  planning  for  major  additions 
to  the  system  was  accomplished,  and  a  sensible  approach  to  acquiring 
them  was  established.  Major  additions  after  initial  operation  included 
the  frequency  diversity  radars,  the  century  series  interceptors,  a  capa¬ 
bility  to  track  supersonic  aircraft,  the  BOM  ARC  and  NIKE  surface- 
to-air  missiles  and  the  airborne  long  range  radar  input  (ALRI)  system. 

The  emphasis  on  acquiring  subsystems,  as  opposed  to  capabilities, 
that  one  finds  in  today’s  approach  to  system  acquisition  makes  broad 
use  of  preplanned  evolutionary  development  and  acquisition  more 
difficult.  Because  MITRE  has  a  role  in  so  many  of  the  C^I  subsystems 
purchased  by  the  government,  the  Corporation  has  a  special  responsi¬ 
bility  for  ensuring  that  the  subsystems  may  be  effectively  combined  to 
achieve  the  required  capability.  As  discussed  above,  MITRE’s  exper¬ 
tise  on  the  technical  characteristics  of  C^I  systems  can  be  applied  to 
minimize  problems  between  interfacing  systems. 

Beyond  that,  even  in  individual  acquisition  programs,  there  are 
important  considerations  of  evolutionary  development.  One  need 
only  examine  the  history  of  the  Airborne  Warning  and  Control 
System  (AWACS)  program  to  appreciate  the  number  of  important 
additions  to  the  system’s  capability  that  have  occurred  over  the 
reasonably  short  life  of  that  program.  Again,  there  are  important 
technical  considerations  that  MITRE  can  help  identify  and  evaluate. 


Development  implies  un¬ 
certainty  and  uncertainty  in  ony 
dimension  implies  uncertainty 
in  cost.  A  fixed-price  contract 


may  limit  the  government's 


cost  exposure  but  moy  provide 


less  performance  then  required. 


Knowing  that  future  additions  will  be  made,  perhaps  one  should 
purchase  more  computer  capacity  than  is  required  for  the  first  incre¬ 
ment  of  capability,  thereby  avoiding  a  costly  future  computer  retro¬ 
fit.  Or  perhaps  the  softw'are  architecture  can  be  partitioned  to 
facilitate  future  known  additions.’*'  It  is  important  for  .MITRE  to 
consider  such  alternatives  as  part  of  its  work  in  the  precontractual 
phase  of  the  acquisition  program,  and  to  provide  its  assessments  to 
the  SPO  in  a  timely  fashion. 

As  another  consideration  in  helping  to  map  the  acquisition  strategy 
of  a  new  program,  MITRE  must  consider  evolutionary  development 
and  make  recommendations  for  its  application.  Perhaps  one  piece  of 
the  capability  will  take  considerably  longer  than  others.  Maybe  a 
useful  capability  can  be  achieved  significantly  earlier  if  a  phased 
implementation  is  employed.  This  is  another  important  area  in  which 
the  Corporation’s  knowledge  and  experience  can  help  make  an  im¬ 
portant  contribution  to  a  successful  program. 

Another  aspect  of  acquisition  strategy  concerns  the  type  of  con¬ 
tract  to  be  employed.  Again,  one  approach  may  go  in  and  out  of 
fashion  as  people  change  or  as  the  community  struggles  with  how  to 
improve  the  acquisition  process.  Contract  type  is  indeed  an  important 
choice,  but  it  is  not  a  panacea.  First,  one  must  remember  that  the 
objective  is  achieving  the  required  capability.  If  one  saves  a  great  deal 
of  money  but  does  not  get  the  capability,  the  program  has  failed. 
Beyond  that,  contract  type  does  not  even  guarantee  that  money  will 
be  saved.  A  few  observations  relevant  to  contract  type  can  be  made. 
They  grow  out  of  MITRE’s  experience  across  many  C^I  system  acqui¬ 
sitions.  First,  true  development  efforts  cannot  be  accomplished  for  a 
fixed  price.  By  its  nature,  development  implies  uncertainty  and  uncer¬ 
tainty  in  any  dimension  implies  uncertainty  in  cost.  One  may  use  a 
fixed  price  contract  to  limit  the  government’s  cost  exposure,  but  in 
that  case,  one  has  to  be  prepared  to  live  with  a  product  that  may 
provide  less  performance  than  required. 


-’B.M.  Horowitz,  The  Importance  of  Architecture  m  DOD  Software,  M91-,t,5,  The 
MITRE  Corporation,  Bedford,  MA,  July  1991. 


Incentive  contracting  is  another  approach  frequently  used.  The  con¬ 
tractor  is  given  special  incentives  to  contain  the  cost  by  being  required  to 
pay  for  any  cost  overruns  but  allowed  to  share  in  any  cost  underruns. 

The  contractor  may  receive  extra  fee  if  the  required  performance  is 
exceeded  or  be  penalized  if  it  falls  short.  Alternatively,  the  fee  may  be 
varied  as  a  function  of  the  government’s  judgment  of  contractor  perfor¬ 
mance — the  so-called  award  fee.  Experience  indicates  that  such  incentives 
may  work,  but  they  also  may  be  ineffeaive  or  even  counterproductive. 

When  the  contractor  has  signed  up  for  too  low  a  price,  a  cost  incentive 

may  detract  from  performance.  The  trouble  is  that  the  incentives  may  not  j 

represent  the  major  drivers  that  govern  the  conduct  of  the  contractor.  ' 

Contioct  type  is  nevet  on 

Other  important  factors  may  include  reputation,  establishing  the  corpo¬ 
ration  in  a  new  business  area,  deferring  profit  to  a  subsequent  program  occeptoble  substitute  for 

phase,  and  the  desire  to  hold  a  team  together  or  to  keep  a  facility  open. 

4^  good  monogement. 

However,  the  most  important  point  is  that  the  use  of  incentive  contracts 

does  not  relieve  either  MITRE  or  the  government  from  the  responsibili¬ 
ties  to  carefully  monitor  program  progress  and  to  actively  pursue  any 
changes  deemed  necessary.  Contract  type  is  never  an  acceptable  substi¬ 
tute  for  good  management. 

In  formulating  the  acquisition  strategy  a  few  other  concerns  should  be 
addressed.  Multiple  concurrent  development  contracts  should  be 
avoided.  Often,  they  will  get  out  of  synchronization  in  both  time  and 
performance,  will  cost  time  and  money,  and  may  require  a  compromise 
in  the  achieved  capability.  It  will  create  a  management  problem  in  resolv¬ 
ing  which  contract  should  be  modified  and  how.  For  example,  it  is  highly 
risky  to  undertake  a  software  development  effort  until  the  computer  on 
which  the  software  will  operate  has  completed  development. 

One  final  comment  on  acquisition  strategy  is  appropriate.  The 
approach  should  not  demand  too  much  too  early.  The  front  end  of 
the  program  should  provide  time  for  the  contractor  to  gain  a  good 
understanding  of  the  requirements,  do  the  necessary  system  design 
work,  and  get  agreement  with  the  government  on  the  design 
approach.  Inadequate  time  up  front  will  cause  pressures  that  lead  to 
bad  decisions.  Everyone  will  want  to  maintain  the  schedule.  As  an 


i 
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example,  when  schedules  are  tight,  some  people  will  want  to  proceed 
with  coding  the  computer  programs,  even  though  the  design  is  incom¬ 
plete.  If  properly  used,  adequate  time  spent  at  the  beginning  will 
result  in  substantial  savings  later  in  the  program.  .MITRE  should 
work  to  ensure  that  the  acquisition  program  does  not  demand  too 
much  too  early.  The  staff  should  then  work,  .vith  the  government  and 
the  contractor  to  ensure  effective  use  of  the  time  provided. 

The  key  questions  to  be  asked  in  reviewing  the  proposed  system 
acquisition  strategy  are:  Does  the  strategy  accurately  reflect  the  state  of 
development  as  it  applies  to  the  desired  capability?  Is  the  implied  distri¬ 
bution  of  responsibilities  among  the  contractor,  government,  and 
MITRE  optimum  for  achieving  the  required  capability?  Are  the  time 
and  money  allotted  consistent  with  the  effort  to  be  accomplished?  Does 
the  plan  provide  for  adequate  access  to  the  contractor  information 
necessary  for  effective  government  program  management? 

System  Design  and  Spedfkotien 

The  most  important  paper  produced  by  MITRE  in  the  system 
engineering  role  is  the  system  specification.  This  document  is  an 
essential  part  of  the  procurement  package  used  to  solicit  industry 
proposals.  In  large  measure,  the  industry  responses  are  evaluated  in 
terms  of  their  responsiveness  to  the  MITRE  specification.  Throughout 
the  development  program,  the  specification  is  used  as  a  guide  against 
which  to  make  judgments  on  progress  or  to  evaluate  proposed 
changes.  The  specification  describes  the  minimum  essential  technical 
requirements  and  sets  the  boundaries  for  the  system  to  be  acquired. 

As  with  the  other  subjects  discussed  herein,  this  section  does  not 
attempt  to  provide  a  step-by-step  description  on  how  to  w’rite  a 
system  specification.  Instead,  the  material  describes  the  role  of  the 
system  specification  and  identifies  some  of  the  major  factors  that 
must  be  considered  by  MITRE  in  preparing  it.  Other  MITRE  docu¬ 
ments  discuss  the  preparation  of  a  system  specification.  '"-^' 

R.S.  Nielsen,  A  Purposeful  Specification,  WP  3750,  The  MITRE  Corporation, 
Bedford,  MA,  1971. 

"  J.W.  Armstrong,  Some  Guidelines  for  Writing  Military  Specifications.  WP  20726, 
The  MITRE  Corporation,  Bedford,  MA,  1976. 


MITRE  staff  must  keep  one  ultimate  objective  in  mind  when 
preparing  the  system  specification;  an  operational  system  that  satisfies 
the  government’s  needs.  In  effect,  when  MITRE  prepares  a  system 
specification  and  provides  it  to  the  government,  the  Corporation  is 
saying;  If  the  system  satisfies  the  specification,  it  will  meet  the 
government’s  stated  operational  requirements.  Although  not  explicitly 
expressed  in  this  way,  the  system  specification  carries  with  it  a 
MITRE  guarantee  of  this  form.  When  approved  by  the  many  govern¬ 
ment  agencies  that  review  it,  the  specification  becomes  the  technical 
yardstick  against  which  contractors  are  selected  and  future  progress  is 
measured.  After  preparing  the  system  specification,  MITRE’s  system 
engineering  responsibility  then  consists  of  helping  the  government,  in 
every  way  possible,  to  acquire  a  system  that  satisfies  the  requirements 
established  in  the  approved  system  specification. 

One  aspect  of  the  system  specification  coordination  process  de¬ 
serves  special  mentioii  here.  In  their  book  on  systems  engineering, 
C.D.  Flagle  et  al.  note, 

A  new  scientific  hypothesis  or  discovery  is  considered  valid  only 
after  it  has  been  subjected  to  scrutiny  and  after  its  truth  has  been 
verified.  A  new  design,  however,  is  seldom  exposed  to  such 
treatment.  Since  the  design  is  more  of  an  art  than  it  is  a  science, 
its  evaluation  by  others  tends  to  be  subjective.^^ 

This  quote  is  especially  true  in  the  stage  when  the  system  specifica¬ 
tion  is  under  review.  MITRE  must  be  prepared  to  defend  its  choice  of 
technical  requirements  by  being  able  to  tie  them  back  to  operational 
requirements  and  by  whatever  analysis  and  experimentation  may  be 
necessary  to  provide  evidence  of  their  validity. 

The  process  involved  in  starting  a  program  to  acquire  a  new  op¬ 
erational  mission  capability,  or  to  make  a  major  addition  to  an  exist¬ 
ing  one,  can  be  long  and  tortuous.  That  process  may  delay  the  needed 
capability  for  years.  Naturally  enough,  once  an  acquisition  program 
is  finally  approved,  there  is  a  great  interest  on  all  sides  in  achieving 


If  the  system  sotisfies  the 
specification,  it  will  meet 
the  government's  stoted 
operotionol  requirements. 


’’C.D.  Flagle,  W.H.  Huggins,  and  R.H.  Roy,  Operations  Research  and  System 
Engineering  (Baltimore:  The  Johns  Hopkins  Press),  I960,  p.  107. 


Competitive  industry  develops 
outstanding  technology 
opplicoble  to  C^l  systems. 

On  the  other  bond,  the 
government  is  not  seeking 
innovation  in  every  dimension 
of  0  new  copability.  The 
government  does  not  need  it 
ond  connot  afford  it. 


the  required  capability  as  soon  as  possible.  Everyone  wants  to  get  the 
procurement  package  together  and  hire  the  contractor  to  build  the 
system.  However,  at  the  front  end  of  the  program,  one  must  spend 
the  time  and  money  necessary  to  provide  a  solid  foundation  in  re¬ 
quirements,  technical  approach,  acquisition  strategy,  and  schedule 
and  cost  estimates.  Judgments  are  required  on  how  much  is  enough, 
and  MITRE  must  be  prepared  to  provide  the  SPO  with  advice  in 
these  areas — both  to  help  ensure  that  what  is  prudent  to  do  is  accom¬ 
plished  and  to  help  avoid  unnecessary  delays  or  expenditures.  To 
avoid  unnecessary  delay  after  a  program  is  approved,  and  assuming 
the  necessary  resources  can  be  made  available,  uncertainties  and  risk 
areas  should  be  pursued  during  the  period  in  which  the  government  is 
deciding  whether  to  proceed  with  the  system. 

Competitive  industry  develops  outstanding  technology  applicable 
to  C^I  systems.  The  acquisition  process  must  provide  for  appropri¬ 
ately  applying  that  technology.  On  the  other  hand,  the  government  is 
not  seeking  innovation  in  every  dimension  of  a  new  capability.  The 
government  does  not  need  it  and  cannot  afford  it.  After  building 
systems  of  a  certain  kind  for  a  number  of  years,  one  begins  to  under¬ 
stand  what  level  of  performance  in  certain  functions  is  adequate  to 
support  required  operational  capabilities,  even  when  those  capabili¬ 
ties  have  to  be  extended  to  some  degree.  That  is,  in  some  cases,  the 
government  and  MITRE  understand  the  performance  that  a  certain 
design  can  achieve  and  have  established  that  level  of  performance  as 
adequate  for  the  capability  intended  for  acquisition.  To  give  an  ex¬ 
ample,  consider  the  design  of  the  computer  program  that  accepts 
radar  data  and  uses  it  to  establish  and  follow  aircraft  tracks.  This 
function  is  referred  to  as  automatic  tracking.  A  tracking  system 
design  that  provides  adequate  performance  for  a  certain  class  of 
systems  has  existed  for  over  25  years.  Systems  using  that  design  have 
operated  very  successfully  for  many  years.  However,  since  system 
design  is  the  responsibility  of  industry,  the  government  does  not 
prescribe  design  information  as  part  of  the  specification  provided  to 
industry  in  soliciting  industry  bids. 


At  the  same  time,  the  acquisition  community  recognizes  that  perti¬ 
nent  design  information,  often  developed  for  the  government  at  consid¬ 
erable  expense,  should  be  made  available  to  the  industry  for  its  use  in 
building  systems  that  operate  as  required  and  are  affordable  both  in 
their  acquisition  and  operating  costs.  Over  time,  alternative  approaches 
have  been  developed  to  provide  design  information  to  industry  without 
including  it  in  the  system  specification  and,  therefore,  without  requir¬ 
ing  that  it  be  applied  to  the  new  system  being  acquired.  Strawman 
designs  developed  in  feasibility  studies  by  MITRE  or  industry  can  be 
made  available  for  information  purposes  in  a  library  accessible  to  all 
potential  bidders.  Some  information  may  be  included  in  the  formal  bid 
package  for  information  only  in  “notes  to  the  specification.”  Informa¬ 
tion  exchange  meetings  can  take  place  prior  to  the  formal  bidding 
period.  To  ensure  that  industry  has  appropriate  design  approaches,  the 
bid  package  may  require  that  the  industry  response  include  feasibility 
designs  for  critical  areas. 

On  the  matter  of  design  versus  performance  information,  the  ap¬ 
proach  MITRE  must  follow  is  clear.  Minimally  acceptable  performance 
requirements  must  be  included  in  the  specification  because  they  tell  all 
parties  what  must  be  achieved.  At  the  same  time,  the  Corporation  must 
acquire  or  develop  enough  design  level  information  to  establish  system 
feasibility  and  to  successtu.ly  estimate  potential  system  costs  and  sched¬ 
ules.  The  Corporation  should  then  work  with  the  SPO  team  to  make 
appropriate  portions  of  this  information  available  to  industry  in  ways 
that  neither  stifle  industry  initiative  nor  open  the  government  to  undue 
risk.  That  risk  is  a  double-edged  sword.  The  government  wishes  to 
minimize  the  risk  of  system  failure  and  therefore  has  an  incentive  to 
provide  all  the  information  it  has  to  the  industry.  The  government  also 
wishes  to  avoid  the  risk  of  future  claims  that  design  information  pro¬ 
vided  by  the  government  was  faulty.  For  any  particular  system,  there  will 
be  a  tradeoff  in  these  two  objectives,  and  the  amount  of  design  informa¬ 
tion  provided  to  the  industry  will  revolve  around  the  depth  of  the 
government’s  experience  in  an  area  and  the  extent  to  which  the  new 
system  requires  significant  increases  in  the  state  of  the  art. 


Obviously,  MITRE  has  neither  the  knowledge  nor  the  resources  to 
specify  a  complete  design  in  sufficient  derail  for  the  contractor  to 
build  the  system  without  going  through  another  design  phase.  That 
is,  MITRE  is  not  in  the  business  of  providing  “build-to”  specifica¬ 
tions  for  C^I  systems.  As  in  so  many  areas  of  systems  engineering, 
informed  judgments  must  be  made  by  MITRE  in  recommending  how 
much  design  information  to  provide  to  industry.  As  noted  by  W.R. 
Beam, 

The  most  successful  system  designers  usually  create  designs 
evolved  from  proven  architectural  approaches  rather  than  seek¬ 
ing  change  for  the  sake  of  change.  *  * 

When  a  function  is  very  critical  and  a  working  design  is  known, 
the  design  should  be  provided.  When  certain  design  approaches  are 
known  to  be  flawed,  they  should  be  ruled  out.  Every  good  engineer 
likes  to  design.  MITRE  staff  must  control  that  appetite  and  restrict 
their  inclusion  of  design  approaches  to  those  known  as  critical  to 
system  performance  and  to  those  well-proven  in  comparable  systems. 
Even  when  design  approaches  are  provided  in  a  bid  package,  the 
proposal  evaluation  process  must  allow  for  industry  to  submit  alter¬ 
native  designs  if  they  can  demonstrate  they  are  substantially  better  in 
performance  or  lower  in  cost. 

Of  course,  MITRE  must  also  avoid  “goldplating”  either  the  system 
requirements  or  the  system  design.  Again,  quoting  Beam, 

The  novice  system  designer  may  he  worried  if  a  requirement 
does  not  address  all  design-decision  topics.  The  well-qualified 
system  designer  views  overrequirement  as  an  evil  and  wel¬ 
comes  those  that  permit  maximum  leeway  in  the  design.  If  the 
requirements  appear  to  be  delineated  either  through  insuffi¬ 
cient  or  overrequirement,  the  experienced  designer  will  take 
steps  to  understand  the  needs  of  the  user(s)  and  establish 
priorities.^'* 


"W.R.  Beam,  Systems  Engineering  Architecture  and  Design  (New  York:  .McGraw- 
Hill,  1990)  p.  86. 

"  Beam,  p.  54. 


Too  much  or  too  little  in  either  requirements  or  design  approach 
leads  to  systems  that  fail  to  provide  the  required  capability  or  that  do 
so  only  after  unreasonable  delays  and  expenditures.  Analogously, 
MITRE  must  avoid  the  premature  application  of  technology  in  pre¬ 
paring  the  system  specification  and  must  help  the  government  pre¬ 
clude  industry  attempts  to  introduce  it  when  the  risks  or  the  costs 
associated  with  it  are  unnecessary. 

In  establishing  proposed  technical  requirements,  MITRE  must 
focus  on  those  user  requirements  that  have  been  most  clearly  identi¬ 
fied.  One  should  avoid  including  requirements  just  because  they 
were  part  of  an  earlier  system.  Requirements  for  “off-the-shelf,” 
“modular,”  and  other  like  slogans  must  be  used  with  great  care. 
They  should  not  be  invoked  unless  they  can  clearly  be  satisfied. 
Wholesale  application  of  military  specifications  (MILSPECS)  is 
expensive  and  may  be  counterproductive.  MITRE  must  be  conver¬ 
sant  with  the  details  of  these  specifications  and  apply  them  selec¬ 
tively  and  sparingly  in  consonance  with  what  is  really  necessary  to 
ensure  achieving  required  system  performance.  The  Corporation 
must  be  prepared  to  defend  those  choices  against  others  who  will 
want  to  err  on  the  conservative  side  by  more  widely  invoking  these 
MILSPECS. 

MITRE  must  be  concerned  with  how  the  system  to  be  acquired 
will  interact  with  other  existing  and  planned  systems.  An  overall 
architecture  must  be  developed  and  kept  in  mind  as  the  system  speci¬ 
fication  is  prepared.  W.R.  Beam  characterizes  the  desired  architecture 
as  follows: 

The  author  believes  that  an  expertly  architected  system  stands 
out  from  the  ordinary,  in  certain  characteristic  ways.  These  are 
often  readily  discerned,  even  by  diligent  students  of  system 
architecture: 


MITRE  must  ovoid  the 
premoture  opplicotion  of 
technology  in  preporing  the 
system  specificotion  ond  must 
help  the  government  preclude 
industry  ottempts  to  introduce 
it  when  the  risks  or  costs 
ore  unnecessory. 


It  evidences  an  overall  unity — its  parts  do  not  compete  but 
complement  one  another,  and  are  similar  in  quality,  durability, 
and  utility. 


It  has  no  parts  that  appear  to  be  afterthoughts.  Likewise,  there  is 
little  waste  in  its  operation,  no  duplication  of  parts  except  that 
required  to  fulfill  functional,  performance,  or  reliahtlity  objectives. 

It  exhibits  balance,  order,  symmetry  from  many  points  of  view: 
e.g.,  internally  (through  its  structure  and  organization),  exter¬ 
nally  (through  its  appearance  and  ease  of  access  and  use), 
logically  (through  design  relationships),  and  functionally  (through 
economy  of  design,  meeting  the  objectives  without  waste). 

It  has  not  only  a  sound  top-level  scheme  but  its  quality  holds  up 
in  detail  as  well — close  examination  of  its  parts  reveals  the  same 
qualities  and  soundness  as  does  the  system  as  a  whole.  * ' 

The  various  subsystems  that  compromise  the  total  system  must  be 
identified  as  part  of  the  system  specification  and  incorporated  in  the 
MITRE  proof-of-concept  design  work  discussed  below.  Each  of  the 
system  functions  must  be  distributed  to  the  subsystems  and  allotted 
among  hardware,  software,  and  operating  personnel.  One  must  avoid 
optimization  at  the  functional  or  subsystem  level;  it  is  total  system 
performance  that  matters.  And  it  is  not  just  the  C^I  system  level 
either,  but  rather  the  operational  mission  level,  such  as  described 
earlier  in  this  book. 

In  considering  the  overall  architecture,  MITRE  must  identify, 
isolate,  and  control  key  risk  areas.  Some  wif  >  lical  in  nature, 
others  will  be  management-oriented.  If  the  etfons  ...  other  organiza¬ 
tions  are  necessary  to  enable  a  successful  program,  MITRE  must 
identify  them  and  work  as  part  of  the  SPO  team  to  secure  the 
necessary  commitments  from  those  organizations.  During  the  pro¬ 
gram,  MITRE  must  help  monitor  whether  those  commitments  are 
being  satisfied  in  a  timely  way  and  apprise  the  program  director 
accordingly. 

For  technical  risks,  MITRE  must  undertake  risk  reduction  efforts.  In 
some  cases,  the  Corporation  should  recommend  industry,  academia,  or 


government  initiatives  to  reduce  the  risk.  These  activities  may  be  part  of 
the  development  contraa  or  they  may  be  accomplished  under  separate 
contracts  before  initiating  the  formal  acquisition  program.  Risk  reduc¬ 
tion  analyses  and  experimentation  may  also  be  conducted  by  MITRE.  In 
any  case,  the  results  should  be  reflected  in  the  system  specification.  As  a 
rule,  if  one  knows  how  to  reduce  important  risks,  the  approach  should 
be  specified.  Any  significant  residual  risk  in  performance,  schedule,  or 
cost  should  be  taken  into  account  by  tailoring  the  acquisition  strategy,  as 
discussed  in  the  previous  section. 

One  important  question  that  MITRE  must  answer  in  preparing  the 
system  specification  is  whether  there  is  a  conceivable,  implementable 
system  design  that  would  satisfy  the  performance  requirements  estab¬ 
lished  in  the  specification.  Having  outlined  the  performance  require¬ 
ments,  MITRE  must  examine  them  and  postulate  an  initial  system 
design  to  meet  them.  This  was  referred  to  above  as  a  proof-of-con- 
cept  design.  It  is  a  reasonableness  test  of  the  technical  performance 
requirements.  It  helps  to  identify  risk  areas,  which  should  be  studied 
as  early  as  possible.  The  requirements  should  be  steered  in  the  direc¬ 
tion  of  lowest  risk. 

Identification  of  major  cost  drivers  is  another  important  product  of 
initial  MITRE  design  efforts.  Pre-acquisition  contract  work  by  MITRE 
or  by  industry  may  be  necessary  to  establish  ways  of  accomplishing  the 
required  functions  at  lower  costs.  The  high  cost  of  certain  functions 
may  result  in  a  reexamination  of  the  system  requirements.  Cost  consid¬ 
erations  could  dictate  a  change  in  the  distribution  of  functions  among 
the  hardware  and  software  components  of  the  system,  or  that  some 
functions  planned  for  support  by  the  data  processor  are  best  accom¬ 
plished  by  the  system  operators.  Like  technical  risks,  high  cost 
functions  must  be  examined  closely  to  be  sure  they  reflect  important 
requirements  and  that  the  approach  to  achieving  them  is  a  good 
balance  between  what  is  needed  and  what  it  costs. 

Since  a  C’l  system  will  evolve  and  may  have  to  operate  for  many 
years  into  the  future,  growth  provisions  are  another  concern  to  be 
addressed  in  the  system  specification.  Again,  informed  judgment  is 


I 
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required  to  provide  prudent  growth  capability  without  unduly  in¬ 
creasing  the  cost  of  the  initial  system  acquisition.  In  addition  to 
growth,  one  might  provide  the  necessary  interfaces  to  accommodate 
other  systems  that  will  follow  in  development. 

Considerations  of  the  human  operator  are  another  very  important 
factor  in  initial  system  specification  and  design  work.  The  distribu¬ 
tion  of  functions  among  the  operators  and  the  system  hardware  and 
software  is  critical.  Operators  must  remain  in  control  of  the  system. 
At  the  same  time,  operator  reaction  times  should  not  seriously  de¬ 
grade  system  performance.  One  should  be  careful  to  avoid  underesti¬ 
mating  what  the  operating  personnel  can  contribute  to  a  successful 
system.  Placing  too  much  reliance  on  the  automated  portion  of  the 
system  can  be  very  costly.  The  burden  of  tedious,  repetitive,  and 
boring  tasks  should  not  be  placed  on  the  operators.  At  the  other 
extreme,  one  cannot  demand  that  all  the  operators  be  the  equivalent 
of  Ph.Ds.  Careful  consideration  should  also  be  given  to  how  the 
operators  will  work  with  one  another.  Whenever  people  are  involved, 
special  care  must  be  given  to  life  support  functions.  All  of  these 
matters  have  to  be  addressed  as  part  of  preparing  the  system  specifi¬ 
cation.  References  to  MITRE  documentation  relevant  to  the  user 
interface  design  may  be  found  in  the  Appendix. 

In  the  process  of  preparing  the  system  specification,  MITRE  will 
identify  a  number  of  requirements  for  the  system  being  acquired  to 
interface  with  other  existing  or  planned  systems.  The  number  of  these 
interfaces  can  be  very  large.  For  example,  a  system  such  as  AW  ACS 
must  interface  with  many  different  U.S.  and  allied  ground  C’l  sys¬ 
tems.  It  also  must  interface  with  many  different  aircraft  and  with 
Navy  surface  ships.  The  interface  characteristics  of  any  new  system 
must  be  specified.  Typically,  MITRE  must  also  be  prepared  to  moni¬ 
tor  the  evolution  of  interfacing  systems  to  ensure  that  when  changes 
are  made  in  other  systems  they  are  reflected  in  the  system  being 
acquired  and  for  which  the  Corporation  has  systems  engineering 
responsibility.  In  some  cases,  the  new  system  requirements  and  the 
initial  design  will  suggest  that  changes  should  be  made  in  one  or 


more  of  the  interfacing  systems.  MiTRE  must  identify  those  instances 
and  work  with  the  SPO  to  get  them  negotiated  with,  and  imple¬ 
mented  by,  those  responsible  for  the  other  systems. 

Another  key  aspect  of  MITRE’s  work  in  preparing  the  system 
specification  concerns  how  one  can  measure  progress  toward  satisfy¬ 
ing  the  requirements  of  the  system  specification  and  in  turn  demon¬ 
strate  that  the  required  system  capability  has  been  achieved.  Quality 
assurance  provisions  must  be  included  in  the  system  specification. 
They  include  requirements  for  system  performance  data  recording, 
data  reduction,  and  analysis  capabilities.  The  testing  provisions  of  the 
system  specification  are  further  detailed  in  other  documentation,  as 
discussed  in  the  Operational  Testing  section. 

Some  key  questions  must  be  answered  during  the  preparation  and 
coordination  of  the  system  specification.  Does  the  specification 
describe  a  system  that  will  satisfy  the  stared  user  requirements.^  Do  all 
parties  agree  on  that.’  Can  the  technical  requirements  in  the  system 
specification  be  traced  back  to  the  user’s  operational  requirements? 
Have  the  major  risks  been  identified?  Have  they  been  reduced  to 
acceptable  levels,  or  is  there  a  plan  to  do  so?  Is  there  at  least  one 
system  design  that  matches  the  specification  requirements  and  can  be 
built  by  industry  within  the  cost  and  schedule  constraints?  Have  all 
the  actions  that  must  be  taken  by  the  government  to  complement  the 
industry  work  been  identified? 

Request  for  Proposal  Pockoge 

The  package  of  material  sent  to  industry  in  selecting  a  contractor 
to  design  and  build  a  new  system  is  known  as  a  request  for  proposal, 
or  RFP.  In  cases  where  MITRE  is  the  system  engineer,  the  RFP  pack¬ 
age  contains  the  MlTRE-prepared  system  specification  as  a  major 
ingredient.  The  specification  describes  the  technical  performance 
requirements  for  the  system.  However,  the  RFP  contains  a  number  of 
other  important  documents  and  the  Corporation  plays  a  significant 
role  in  generating  some  of  them.  These  include  the  statement  of  work 
(SOW),  instructions  to  the  bidders,  contract  data  requirements  list 


(CDRL),  and  perhaps  others,  such  as  the  listing  of  government- 
furnished  support  resources. 

As  the  pieces  of  the  RFP  package  are  prepared,  a  draft  version 
should  be  made  available  by  the  government  to  industry  in  some  way 
that  shows  no  bias  toward  any  particular  contractor.  For  example, 
they  can  be  made  available  in  a  library  that  contractors  may  visit  with 
the  understanding  that  it  is  for  information  purposes  only  and  that  the 
draft  documents  are  subject  to  change  before  final  publication. 

In  preparing  the  RFP,  one  should  attempt  to  minimize  the  number 
of  options  that  a  contractor  must  provide.  They  are  expensive  to 
generate  and  difficult  to  evaluate.  Another  area  that  should  be 
watched  closely  is  the  requirement  for  the  contractor  to  submit  docu¬ 
mentation  to  the  government.  Documentation  is  very  expensive  to 
produce  initially  and  very  costly  to  review  by  the  government  and 
maintain  by  the  contractor.  Documentation  requirements  should  be 
kept  to  a  minimum.  Every  document  required  should  have  a  clear  and 
necessary  purpose. 

The  government’s  commitments  on  the  program  must  be  accu¬ 
rately  described  and  the  responsible  agencies  clearly  identified.  The 
government’s  intentions  for  the  various  test  phases  must  be  enumer¬ 
ated,  and  procedures  for  retesting  and  for  change  control  should  be 
included. 

The  plans  by  which  the  source  selection  will  be  conducted  should 
be  provided  and  evaluation  criteria  identified.  If  specific  data  or 
demonstrations  or  sample  systems  are  required,  they  must  be  care¬ 
fully  described  and  their  role  in  the  evaluation  process  discussed. 

Many  sections  of  the  RFP  will  be  prepared  by  the  government 
itself.  MITRE  p;epares  the  system  specification  and  ..dps  with  many 
of  the  other  sections.  As  system  engineer,  the  Corporation  must 
review  the  entire  RFP  package  to  ensure  that  it  describes  the  capabil¬ 
ity  the  government  requires,  its  provisions  are  internally  consistent, 
and  it  provides  a  reasonable  basis  for  contractor  selection  and  subse¬ 
quent  system  development.  The  key  questions  to  be  answered  are 
embodied  in  these  considerations. 


PropMoi  IvfliiNrtiM 

MITRE  has  published  an  excellent  document  on  the  source  selec¬ 
tion  process,  corporate  policies  and  procedures  relating  to  the  partici¬ 
pation  of  personnel  in  that  process,  and  the  attendant  ethical 
considerations  and  issues.^*  Suffice  it  to  say  that  source  selections  are 
very  important  government  responsibilities.  MITRE  must  be  vigilant 
in  all  it  does  to  avoid  any  action  that  would  compromise,  or  appear 
to  compromise,  that  process.  The  Corporation’s  credibility  in  the 
systems  engineering  role  for  the  government  is  dependent  on  unbiased 
and  professional  conduct  by  all  its  personnel. 

Since  there  is  such  an  excellent  reference,  there  is  no  need  in  this 
book  to  discuss  MITRE’s  activities  leading  up  to  the  source  selection,  or 
those  that  take  place  during  the  evaluation  of  contractor  proposals  and 
contract  award.  It  should  be  noted,  however,  that  MITRE  does  not  vote 
on  which  contractor  should  be  awarded  the  contract.  The  Corporation 
does  provide  an  evaluation  of  all  contractor  technical  proposals  against 
the  requirements  of  the  MITRE  system  specification  included  in  the  REP 
package.  Also,  although  MITRE  does  not  have  access  to  the 
contractor’s  cost  proposals,  MITRE  staff  assigned  to  the  source  selec¬ 
tion  team  do  provide  evaluations  of  the  amount  of  effort  estimated  by  a 
contractor  to  complete  a  given  technical  task.  It  takes  a  technical  person 
to  know  how  much  effort  a  technical  task  is  apt  to  require. 

In  helping  the  government  formulate  the  REP  package  and  evaluate 
contractor  responses,  MITRE  should  search  for  ways  that  will  most 
readily  indicate  the  contractor’s  understanding  of  the  work  to  be 
done.  They  should  be  included  in  the  REP  and  used  in  the  evaluation. 
Special  care  should  be  taken  to  review  areas  in  which  the  specifica¬ 
tion  is  not  detailed. 

Often,  after  all  the  work  of  proposal  evaluation  is  complete,  the 
government  may  go  into  final  negotiations  with  one  or  more  contrac¬ 
tors.  Historically,  since  costs  are  often  a  major  issue  in  those  negotia¬ 
tions,  MITRE  has  often  been  excluded  from  these  final  negotiations. 
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Government  negotiators  are  not  trained  in  the  technical  disciplines 
involved  in  the  systems.  Because  of  that,  agreements  may  be  negoti¬ 
ated  in  ways  that  negatively  affect  the  technical  performance  of  the 
proposed  system.  MITRE  staff  should  play  a  role  in  the  final  negotia¬ 
tions.  Experience  has  shown  that  participation  by  skilled  technical 
personnel  can  avoid  compromises  in  required  operational  capability, 
and  in  some  other  cases  has  actually  saved  the  government  consider¬ 
able  sums  of  money.  If  one  works  at  it,  MITRE  can  support  the 
negotiation  process  without  having  access  to  the  actual  contractor 
cost  proposals. 

There  is  only  one  important  question  to  be  answered  here:  Which 
contractor  will  do  the  best  job — including  performance,  cost,  and 
schedule — of  providing  a  system  that  satisfies  the  requirements  of  the 
REP  and  is  most  likely  to  provide  the  necessary  system  capability? 


participation  by  skilled  tech-  **«*“""« 

This  section  reflects  on  the  MITRE  systems  engineering  activities 

(licol  personnel  con  ovoid  place  during  the  phase  of  the  acquisition  program  in  which 

.  .  .  ,  the  contractors  are  designing,  developing,  building,  and  testing  the 

compromises  in  required 

systems  they  have  been  contracted  to  provide.  For  the  moment,  as- 
operational  capability  and  con  sume  that  the  government  has  contracted  with  a  single  industrial 

corporation  that  will  provide  the  required  system.  MITRE’s  roles  in 

save  the  government 

preparing  the  system  specification  and  in  supporting  the  source  selec- 
considerable  money.  tion  process  are  technically  challenging  activities  to  the  staff.  The 
impact  of  their  efforts  on  the  program  is  direct  and  immediate.  The 
importance  of  these  efforts  is  readily  appreciated  by  all  program 
participants. 

Although  not  as  well  understood  and  accepted,  MITRE’s  work 
during  the  contractor  development  and  acquisition  phases  is  no  less 
important  to  achieving  the  required  system  capability.  Ideally,  that 
work  will  have  a  positive  effect  on  the  success  of  the  program,  will  be 
truly  appreciated  by  both  the  government  and  the  contractor,  and 
will  challenge  the  MITRE  staff.  However,  those  things  will  not  hap¬ 
pen  unless  the  Corporation  continues  to  focus  on  the  capability  that 


is  required  as  the  criteria  for  action.  It  is  especially  important  that 
MITRE  work  with  both  government  and  industry  in  cooperative  and 
supportive  ways.  Some  discussion  of  these  points  may  help  to  clarify 
their  meaning. 

Chapter  3  discusses  the  desired  relationship  between  MITRE  and 
the  government’s  system  program  director.  Similarly,  the  characteris¬ 
tics  of  a  positive  association  between  MITRE  and  industry  are  out¬ 
lined  in  that  section.  Many  aspects  of  the  interactions  between 
MITRE  and  industry  during  the  program  development  phase  are 
discussed.  For  the  purposes  of  the  discussion  here,  it  is  assumed  that 
the  attitudes  and  approaches  described  there  are  operative  on  the 
program.  Special  emphasis  here  is  on  how  the  MITRE  staff  must  view 
its  effort  in  this  phase  for  the  work  to  be  effective  and  satisfying. 

During  the  contractor  development  phase,  the  Corporation  has  the 
responsibility  to  evaluate  the  adequacy  and  reasonableness  of  the 
contractor’s  analysis,  design,  implementation,  and  test  activities.  A 
major  role  of  the  MITRE  project  leader  is  to  formulate  and  control  the 
Corpotation’s  work  in  ways  that  minimize  the  confrontational  nature 
of  the  MITRE  and  industry  relationship,  and  eliminate  unnecessary 
effon  while  making  positive  contributions  to  program  progress.  In 
reviewing  the  contractor’s  technical  progress,  MITRE  staff  must  iden¬ 
tify  critical  technical  progress  and  issues.  The  Corporation  must  work 
with  the  SPO  to  ensure  that  the  industry  provides  the  government  with 
the  necessary  in-progress  information,  and — when  required  by  the 
government — that  the  industry  makes  the  changes  necessary  to  achieve 
the  system  capability.  At  the  same  time,  MITRE  must  avoid  nit-picking 
in  unimportant  areas  and  refrain  from  creating  unnecessary  industry 
effort  and  costs.  To  do  oO,  the  staff  must  be  technically  accomplished 
in  what  it  takes  to  carry  out  the  critical  system  functions  in  the  real 
world,  vigilant  in  assessing  the  progress  of  the  system  acquisition,  and 
effective  in  helping  the  SPO  make  any  necessary  changes  in  the  con¬ 
tractor  work  or  in  the  effort  of  other  program  participants. 

Clearly,  one  major  work  area  involves  the  review  of  contractor 
documentation  and  testing.  To  be  successful  at  it,  MITRE  must 


MITRE  reviews  contractor 
products  to  help  achieve  the 
copobility,  and  only  os  o 
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understand  the  capability  the  government  requires  and  evaluate  the 
contractor  product  in  view  ot  the  contribution  it  is  supposed  to  make 
toward  achieving  that  capability.  The  contractor  has  legal  obligations 
to  the  government  tor  delivery  t)t  products.  MITRE  helps  the  govern¬ 
ment  decide  whether  they  have  been  satistic'^,  but  that  is  not  the  most 
important  reason  tor  the  MITRE  review  of  contractor  products.  The 
Corporation  is  in  business  to  help  the  government  achieve  a  required 
capability.  First  and  foremost,  MITRE  reviews  contractor  products  to 
help  achieve  the  capability,  and  only  as  a  secondary  concern  to  en¬ 
sure  contract  compliance.  There  are  subtle  but  vital  differences  in 
attitude  and  approach  if  the  MITRE  staff  keeps  that  distinction  in 
mind.  These  differences  can  significantly  increase  the  Corporation's 
effectiveness  in  this  phase  of  a  program.  Presumably,  all  organiza¬ 
tions  are  working  to  achieve  the  capability.  Some  specific  approaches 
that  may  be  used  by  MITRE  in  interacting  with  industry  during 
system  development  and  testing  are  described  in  Chapter  .T  They 
emphasize  a  cooperative  effort  among  government,  industry,  and 
MITRE  for  the  achievement  of  the  required  cap.ability. 

Naturally,  this  evaluation  phase  of  the  program  becomes  more 
complicated  if  there  is  more  than  one  corporation  under  contract  to 
the  government  to  provide  a  portion  of  the  required  capability.  All 
the  discussion  above,  and  in  the  sections  referenced,  still  applies.  But 
now  there  is  an  additional  consideration  of  the  interaction  between 
the  efforts  of  the  various  contractors.  Again,  .MITRE  will  face  situa¬ 
tions  in  which  difficult  judgments  must  be  made.  Is  the  best  course  of 
action  to  force  a  contractor  to  comply  with  the  contract  require¬ 
ments.’  Should  the  requirements  be  changed?  Should  one  contractor's 
product  be  modified  to  accommodate  something  that  has  occurred  in 
another's?  The  overriding  criterion  remains  the  same.  What  is  the 
best  thing  to  do  at  this  time  to  achieve  the  required  capability,  all 
things  considered?  Are  there  MITRE?  efforts  that  would  help  the 
contractor?  Those  considerations  include  not  just  technical  matters, 
but  political,  economic,  and  legal  ones  as  well.  These  matters  may 
become  extremely  complex.  They  are  a  challenge  to  the  knowledge. 


ingenuity,  and  statesmanship  of  the  MITRE  staff.  When  viewed  in 
this  light  and  approached  in  this  way,  the  challenge  of  reviewing  the 
contractor’s  progress  is  far  removed  from  the  mundane,  contentious 
drudgery  that  it  can  become  unless  MITRE  and  industry  have  a 
cooperative  approach  in  which  each  respects  the  capabilities  and 
intentions  of  the  other. 

Other  activities  during  the  contractor  design  and  development 
phase  vary  with  the  individual  program.  In  an  evolutionary  develop¬ 
ment  program,  MITRE  may  be  doing  the  studies  and  analyses  leading 

to  preparation  of  the  system  specification  for  the  next  program  incre-  ! 

j 

ment.  The  Corporation  may  be  providing  the  systems  engineering 

Reviewing  contractor  progress  is 

support  required  to  initiate  another  program  that  will  add  to  the 

system  capability.  All  the  factors  that  potentially  dictate  change  challenging  ond  rewording  when 

continue  to  operate  in  the  environment.  The  threat  evolves,  technol- 

MITRE  ond  industry  hove  a 

ogy  developments  proceed,  decisions  are  made  that  affect  funding. 

Progress  is  made  or  delayed  in  other  systems  with  which  the  current  cooperotive  relationship  in 

one  must  interoperate.  The  challenge  to  MITRE  continues:  to  know 

.  .  ,  ,  ,  .  ,  .  ,  ,  which  eoch  respects  the 

in  detail  what  is  happening  throughout  the  environment  in  which  the 

system  is  being  developed;  to  relate  that  to  the  specifics  of  the  current  copobilities  ond  intentions  of 

program  and  especially  to  the  contractor’s  work;  and  to  work  with 

the  government  program  director  to  make  whatever  change  is  neces- 

sary  to  achieve  the  required  capability  on  a  reasonable  schedule  and 

at  a  reasonable  cost.  When  performed  with  this  perspective,  review  of 

contractor  products  has  both  great  challenge  and  high  impact. 

Much  of  this  book  has  concentrated  on  system  performance  and  to 
a  lesser  degree,  on  the  cost  implications  of  the  many  activities  that 
take  place  during  the  acquisition  of  a  major  C^I  system.  However,  as 
has  been  noted  repeatedly,  a  reasonable  schedule  is  also  a  critical 
concern.  In  fact,  in  many  instances  the  success  and  the  cost  of  a 
program  are  driven  in  significant  degree  by  the  schedule.  An  overly 
ambitious  schedule  can  be  just  as  disastrous  as  miscalculations  with 
respect  to  performance  or  cost.  Also,  when  a  particular  activity  is 
delayed,  its  revised  schedule  impacts  not  just  that  work,  but  poten¬ 
tially  many  other  efforts  as  well.  Quite  frequently,  in  mapping  out  a 


remedial  approach  to  a  problem,  the  schedule  will  become  longer 
than  originally  planned.  In  that  case,  one  has  to  decide  whether  to 
announce  the  new  schedule  or  not.  It  is  sometimes  argued  that  if  a 
new  schedule  is  announced,  people  and  organizations  will  not  work 
as  hard  and  as  well  as  they  might  if  the  commitment  to  the  original 
schedule  was  maintained.  The  argument  suggests  that  any  chance  of 
improving  on  the  new  estimate  will  be  lost  by  announcing  it,  or  that 
announcing  a  new  schedule  for  one  activity  may  cause  other  work  to 
be  pursued  less  vigorously  on  the  assumption  that  it  too  will  have 
more  time.  There  is  some  validity  in  these  arguments,  but  there  is  also 
another  side  to  the  question. 

Assume  that  the  user  command  is  to  provide  trained  operational 
and  maintenance  personnel  to  participate  in  operational  evaluation  of 
the  new  system  at  a  certain  date.  Personnel  are  to  be  reassigned  from 
other  activities,  trained,  and  deployed  to  the  test  location.  If  a  system 
development  problem  occurs  that  is  likely  to  delay  the  start  of  system 
testing,  failure  to  inform  the  using  command  of  that  likelihood  could 
lead  to  a  situation  in  which  user  command  personnel  would  be  relo¬ 
cated  and  find  themselves  without  anything  useful  to  do  for  many 
months.  That  is  wasteful  and  obviously  counterproductive  to  a  coop¬ 
erative  attitude  between  the  participants  and  therefore  not  in  the  best 
interest  of  achieving  the  capabilities. 

As  in  most  other  perplexing  matters  of  system  acquisition,  there  is 
no  universal  answer  on  when  to  formally  change  a  schedule.  MITRE 
should  strive  to  help  establish  the  facts  with  respect  to  the  prospective 
schedule,  communicate  the  Corporation’s  assessments,  and  assist  the 
SPO  in  applying  the  results  to  the  current  situation.  MITRE  must 
help  the  SPO  evaluate  the  pros  and  the  cons  in  each  situation.  The 
revised  work  must  be  explicitly  defined,  and  realistic  schedule  and 
cost  estimates  developed  and  negotiated.  A  continuing  series  of  “get 
well”  activities  disillusions  everyone.  Having  established  a  realistic 
schedule,  one  must  then  decide  when  or  if  to  announce  it,  and  to 
whom.  MITRE  should  provide  advice  that  considers  all  related 
factors,  not  just  those  of  a  technical  nature.  Care  must  be  taken  by 


the  government  in  announcing  and  implementing  change  to  avoid 
negating  the  existing  contract  with  industry.  Industry  must  be  party 
to  changes  that  affect  the  performance,  schedule,  or  cost  aspects  of  its 
contract.  Otherwise,  the  existing  contract  may  be  voided  and  require 
complete  renegotiation.  Needless  to  say,  MITRE  must  be  sensitive  to 
considerations  of  this  sort. 

Another  important  activity  for  MITRE  in  this  phase  of  a  program 
involves  helping  the  SPO  respond  to  all  the  requests  for  information 
and  status  reporting  that  always  accompany  an  important  acquisition 
program.  Regular  status  reporting  is  required.  Agencies  both  within 
and  outside  the  DOD  request  information  and  either  give  guidance, 
make  comments,  or  raise  other  questions.  Many  of  these  interactions 
serve  legitimate  management  purposes;  others  are  worthwhile  invest¬ 
ments  in  public  relations.  Whatever  the  objectives,  each  is  important. 
The  information  must  be  factual  and  well  presented.  MITRE  pro¬ 
vides  significant  support  to  the  program  director  in  gathering  the 
necessary  information,  assessing  it,  and  presenting  it  to  those  that 
have  legitimate  access  to  it.  The  effort  may  require  extensive  techni¬ 
cal  work  by  the  Corporation.  It  may  also  be  very  rewarding  when 
the  MITRE  products  are  used  to  influence  the  course  of  the  program 
in  a  positive  way. 

While  the  AWACS  program  was  in  the  contract  development 
phase,  MITRE  staff  members  were  instrumental  in  analyzing  the 
radar  and  communications  performance,  as  well  as  the  survivability 
of  the  AWACS  platform.  This  analysis  helped  convince  a  high  level 
review  committee  that  acquisition  of  the  system  should  be  contin¬ 
ued.  Challenging,  high  visibility  MITRE  efforts  of  this  sort  arise  on 
almost  every  C^I  acquisition  program.  They  can  occur  at  any  point 
in  the  program.  They  are  most  often  initiated  because  someone  or 
some  group  is  uneasy  about  an  aspect  of  the  program.  Helping  the 
program  director  to  overcome  any  such  bias,  and  in  the  process  to 
create  program  supporters,  are  important  MITRE  activities.  The 
visibility  it  affords  is  an  opportunity  for  both  the  Corporation  and 
the  individuals  involved. 


To  summarize  the  key  questions  to  be  addressed  during  this  phase 
of  the  acquisition  program,  consider  the  following:  Are  the  industry 
and  government  activities  proceeding  satisfactorily  toward  the 
achievement  of  the  required  capability?  If  not,  what  needs  to  be 
changed,  and  how?  Similarly,  are  there  events  taking  place  in  the 
environment  surrounding  the  acquisition  program  that  would  dictate 
changes  within  the  program?  What  changes,  and  how  should  they  be 
accomplished?  Are  there  any  activities  that  have  been  overlooked  but 
should  be  initiated?  Are  there  any  unusual  opportunities  that  should 
be  exploited  to  improve  the  program?  Answering  such  questions 
characterizes  the  challenge  to  MITRE  during  this  program  phase 
much  more  appropriately  than  the  somewhat  cynical  phrase  “con¬ 
tractor  monitoring.” 

OpwatioMil  Testing 

There  is  a  large  amount  of  testing  that  goes  on  as  part  of  the 
contractor  development  efforts.  MITRE  is  an  active  participant  in 
those  tests,  helping  to  establish  the  test  requirements  and  reviewing 
the  contractor-proposed  test  program,  test  methods,  data  reduction, 
and  analysis  plans.  The  Corporation  also  is  involved  in  establishing 
the  levels  of  performance  required,  witnessing  tests,  analyzing  data  as 
appropriate,  reviewing  contractor  test  reports,  and  evaluating  the 
results.  When  one  considers  all  the  subsystem  and  system  testing  of 
hardware  and  software,  this  represents  a  major  activity  for  the 
MITRE  staff.  Beyond  this  level  of  testing,  there  are  at  least  two 
others.  The  first  is  a  series  of  system-level  tests  to  determine  whether 
the  product  provided  by  the  contractor  meets  the  requirements  of  the 
system  specification  as  they  are  reflected  in  the  contractor’s  top-level 
design  and  test  documentation.  The  second  level  follows  that  and  is 
aimed  at  determining  whether  the  capability  that  has  been  acquired 
satisfies  the  operational  requirements  of  the  using  command.  It  is  not 
the  purpose  here  to  try  to  describe  all  of  this  in  detail,  but  rather  to 
make  a  few  important  observations  about  the  latter  two  levels  of 
system  testing. 


When  the  government  signs  a  contract  with  industry  to  build  a  CM 
system,  the  industry  is  saying  it  will  provide  a  system  that  meets  the 
government’s  requirements  as  specified  m  the  MfTRE-prenared  sys¬ 
tem  specification.  The  government  is  saying  that  if  the  system  is  built 
as  the  industry  proposes,  it  will  be  satisfactory.  The  top-level  industry 
specification  is  reviewed  by  the  government.  When  the  government 
approves  that  specification,  it  is  agreeing  that  if  the  industry  builds  a 
system  that  meets  the  industry  specification,  it  will  in  turn  meet  the 
requirements  of  the  MITRE-prepared  system  specification  and 
therefore  will  satisfy  the  user  requirements.  The  first  system-level 
testing  is  directed  at  evaluating  whether  the  delivered  system  meets 
the  commitments  made  by  industry  in  their  top-level  system  design 
documentation.  If  so,  the  industry  system  is  accepted  by  the  govern¬ 
ment.  There  then  follows  a  series  of  government-conducted  tests  to 
evaluate  how  well  the  system  performs  its  operational  mission.  It 
should  be  noted  that  changes  to  the  government’s  approach  in  areas 
such  as  when  the  contractor  specification  is  approved  are  now  under 
consideration. 

As  noted,  MITRE  is  extensively  involved  in  the  various  functional 
tests  conducted  as  part  of  the  contractor  development  program.  Most 
of  the  initiative  there  is  on  the  contractor’s  part.  The  contractor 
writes  the  test  plans  and  procedures,  conducts  the  tests,  collects  and 
analyzes  most  of  the  data,  writes  the  reports,  and  submits  them  to  the 
government.  MITRE  reviews  the  contractor  plans,  witnesses  most  of 
the  tests,  and  evaluates  the  results.  However,  for  system-level  tests, 
some  of  the  contractor  initiative  moves  to  MITRE.  Early  in  the  pro¬ 
gram,  the  Corporation  prepares  a  test  and  evaluation  master  plan. 

The  plan  outlines  all  the  testing  that  will  be  done  on  the  system.  For 
system-level  testing,  it  attempts  to  anticipate  any  special  test  equip¬ 
ment,  data  reduction  and  analysis  computer  programs,  personnel,  and 
facilities  that  may  be  required  to  conduct  the  test  programs.  It  estab¬ 
lishes  the  performance  measures  that  should  be  made  and  relates 
them  to  the  requirements  in  the  system  specification.  For  each  test 
measure,  a  level  of  acceptable  performance  is  established.  All  of  this 


One  of  the  most  challenging 
aspects  of  the  system  test  time 
is  to  distinguish  those  things 
that  do  not  work  as  plonned 
from  those  that  work  as 
plonned  but  thot  someone  now 
wishes  to  operate  differently. 


sort  of  information  must  be  coordinated  with  both  the  development 
and  operational  communities. 

For  Air  Force  C^l  programs,  a  second  set  of  system  level  tests  is 
conducted  by  the  Air  Force  Operational  Test  and  Evaluation  Com¬ 
mand,  an  independent  test  agency.  The  tests  are  designed  to  evaluate 
how  well  the  system  performs  the  operational  functions  for  which  it 
was  acquired.  MITRE  is  actively  involved  in  helping  the  test  agency 
to  understand  what  the  system  was  designed  to  do,  how  its  opera¬ 
tional  performance  might  be  measured,  and  what  test  support  will  be 
required  to  carry  out  the  test  program.  The  Corporation  is  then  very 
active  in  witnessing  the  tests  and  evaluating  the  results.  The  test 
agency  makes  its  own  independent  evaluation.  MITRE  helps  the  SPO 
review  that  evaluation. 

The  firsthand  experience  gained  by  MITRE  personnel  in  the  devel¬ 
opment  and  evaluation  testing  is  very  important.  It  provides  a  firm, 
real-world  basis  for  any  work  that  the  Corporation  does  on  the 
current  program.  It  is  also  a  good  learning  experience  that  can  be 
applied  to  the  next  program.  Clearly,  for  programs  developed  in  an 
evolutionary  manner,  knowing  what  exists  is  important  in  deciding 
what  should  be  done  next.  But  even  when  a  system  is  not  evolving, 
understanding  what  it  does  and  how  can  be  extremely  useful  for 
work  on  other  systems  with  which  it  interfaces. 

System  testing  is  the  culmination  of  the  development  cycle.  By  the 
time  it  takes  place,  much  has  happened  that  potentially  changes 
agreements  made  at  the  beginning  of  the  program.  One  of  the  most 
challenging  aspects  of  the  system  test  time  is  to  distinguish  those 
things  that  do  not  work  as  planned  from  those  that  work  as  planned 
but  that  someone  now  wishes  to  operate  differently.  Because  MITRE 
provides  a  large  fraction  of  the  corporate  memory  on  any  long  pro¬ 
gram,  the  Corporation  must  help  the  program  director  to  distinguish 
these  two  problems.  Both  may  occur,  but  unapproved  changes  in 
requirements  should  not  be  the  basis  for  evaluating  how  well  the 
development  community  did  its  job.  This  MITRE  role  requires  expe¬ 
rience,  knowledge,  and  diplomacy. 


As  has  already  been  stated,  there  are  two  key  questions  at  this  stage  of 
a  program:  Does  the  contractor-delivered  product  meet  the  requirements 
of  the  system  specification?  Does  the  system  perform  well  enough  to 
satisfy  the  intended  operational  mission?  In  the  end,  nothing  else  matters. 
The  answers  to  the  above  questions  may,  of  course,  spawn  many  other 
questions.  The  MITRE  role  as  systems  engineer  requires  that  the  Corpo¬ 
ration  be  both  willing  and  able  to  answer  them. 

FitM  Deployaeiit  wd  OperatiM 

MITRE  has  always  prided  itself  on  going  where  the  action  is.  Even 
in  the  very  early  days  of  the  Corporation,  MITRE  personnel  were  in 
residence  at  places  such  as  the  Operational  Air  Defense  Direction 
Center  in  Montgomery,  Alabama.  Over  the  years,  MITRE  people 
have  followed  other  systems  into  the  field,  as  well.  Large  numbers  of 
MITRE  staff  have  been  resident  at  operating  locations  such  as 
NORAD  and  Strategic  Air  Command  headquarters.  When  special 
operational  tests  are  conducted,  MITRE  staff  members  often  attend. 
Activities  of  the  kinds  just  mentioned  have  two  major  payoffs.  The 
immediate  one  is  that  MITRE’s  background  and  experience  in  the 
systems  being  used  can  be  applied  to  helping  the  users  understand  the 
system  and  to  make  maximum  use  of  its  inherent  capabilities.  On  the 
other  side  of  the  coin,  MITRE  learns  a  lot  about  the  command’s 
operational  mission  and  the  limitations  of  their  existing  systems  in 
supporting  those  missions.  That  information  can  be  carried  back  to 
the  development  community  and  reflected  in  the  Corporation’s  subse¬ 
quent  work  for  ESC  in  systems  engineering  of  future  C^I  capabilities. 

MITRE  was  formed  at  the  request  of  the  Air  Force  to  provide 
systems  engineering  on  C^I  acquisition  programs.  On  occasion,  there 
has  been  some  uneasiness  within  the  development  community  about 
MITRE’s  presence  at  operating  locations.  That  uneasiness  has 
stemmed  in  part  from  a  concern  that  there  have  always  been  govern¬ 
ment-imposed  and  natural  limitations  on  the  size  of  the  MITRE  staff. 
Commitment  of  that  staff  to  assist  user  commands  in  exploiting  their 
existing  systems  reduces  the  staff  available  for  systems  engineering  of 


new  CM  capabilities.  The  Corporation  has  always  been  sensitive  to 
those  concerns;  however,  MITRE  strongly  believes  it  is  essential  for  a 
limited  number  of  its  staff  to  work  directly  with  the  using  commands. 
Those  staff  members  can  help  facilitate  the  transition  of  new  systems 
to  the  user,  an  important  job  of  the  development  community.  In 
return,  the  knowledge  of  the  operational  capabilities  gained  by 
MITRE  can  be  applied  to  the  next  systems  engineering  job  for  the 
development  community.  As  noted  many  times  in  this  book,  MITRE’s 
understanding  of  the  Air  Force  operational  mission  capabilities  is  both 

\  unique  within  the  development  community  and  essential  to  the 
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Corporation’s  success  in  the  systems  engineering  role. 

MITRE  strongly  believes  it  is 
essential  for  o  limited  number 
of  its  staff  to  work  directly  with 
the  using  commands. 


MITRE  Staff  at  NORAD  Headquarters 


There  is  one  other  reason  to  encourage  MITRE  staff  to  participate 
in  field  deployments,  field  exercises,  and  field  operations:  it  is  a  good 
reminder  of  why  MITRE  exists — to  help  the  government  achieve 
required  capabilities.  It  is  an  experience  of  unparalleled  dimension  to 
see  what  one  has  worked  so  hard  to  accomplish  operating  as  in¬ 
tended.  This  experience  provides  the  satisfaction  and  encouragement 
required  to  keep  the  Corporation  dedicated  to  helping  in  any  way 
possible  to  provide  the  U.S.  with  the  CM  capabilities  so  vital  to  the 
nation’s  well-being. 


CbapttrThree 


Relotionship  with  Other  Program  Partkipaiits — the  Environmeat 

MITRE  neither  manages  C’l  systems  acquisition  programs  nor 
builds  the  systems  that  result  from  these  programs.  Those  functions 
are  the  province  of  the  government’s  SPO  ..uJ  profit-making  industry, 
respectively.  As  systems  engineer  on  such  programs,  MITRE  works 
closely  with  both  the  SPO  (as  part  of  the  Integrated  Product  Team) 
and  industry.  The  effectiveness  of  the  Corporation’s  relationship  with 
these  key  program  participants  largely  determines  the  extent  to  which 
it  can  contribute  to  the  achievement  of  required  national  capabilities. 
Significant  efforts  have  been  made  over  time  to  ensure  that  the  nor¬ 
mal  role  of  MITRE  as  systems  engineer  on  a  government  acquisition 
program  is  both  appropriate  and  well  understood.  However, 

MITRE’s  role  on  any  given  program,  and  the  extent  of  the  contribu¬ 
tion  that  the  Corporation  is  able  to  make  on  that  program,  are  a 
direct  function  of  the  relationships  with  the  responsible  program 
director  and  the  associated  industry.  This  section  discusses  a  variety 
of  factors  that  may  influence  these  relationships. 

Each  of  the  major  groups  involved  in  a  C^I  system  acquisition 
program — government,  industry,  and  MITRE  when  it  has  the  systems 
engineering  role — exists  and  functions  in  a  very  complicated,  dynamic 


environment  that  from  day  to  day  may  have  profound  impact  on  the 
conduct  and  success  of  the  program.  Understanding  that  environment 
and  helping  to  deal  with  it  as  necessary,  are  major  functions  of  the 
systems  engineer.  The  final  portions  of  this  chapter  summarize  dimen¬ 
sions  of  that  environment  and  MlTRF.’s  approach  to  it. 

MITRE'S  Role  ki  Support  of  tho  Systeoi  Progron  Director, 

Dkoctor  of  Eogiooorwg,  ood  tlM  lotegratad  Prodoct  Teooi 

The  relationship  among  the  system  program  director,  director  of 
engineering,  and  MITRE  project  leader  is  a  critical  factor  in  establish¬ 
ing  the  extent  to  which  the  Corporation  can  contribute  to  the  success 
of  a  major  acquisition  program.  The  relationship  of  MITRE  staff  to 
other  members  of  the  integrated  product  team  is  similarly  important. 
Although  some  of  the  material  in  this  section  describes  the  role  of  the 
MITRE  project  leader,  it  is  important  for  staff  to  understand  the 
attitudes  and  responsibilities  that  pervade  the  government/MITRE 
relationships  when  the  Corporation  assumes  the  systems  engineering 
role.  For  MITRE  to  be  successful,  all  of  the  staff  must  act  in  conso¬ 
nance  with  the  Corporation’s  standards  and  methods. 

There  is  no  item  in  the  federal  budget  that  is  labeled  “MITRE.” 
The  Corporation’s  activity  on  a  program  must  be  justified  by  the 
program  director  and  paid  for  out  of  program  funds.  From  year  to 
year,  MITRE  enjoys  no  guarantee  of  future  funding.  Clearly  then,  the 
Corporation  must  satisfactorily  accomplish  the  work  required  by 
each  program  director  and  director  of  engineering  for  whom  it  pro¬ 
vides  systems  engineering  support.  This  must  be  done  in  ways  that 
make  MITRE  the  best  choice  for  systems  engineering  from  both 
performance  and  cost  perspectives.  MITRE’s  work  on  a  project  is 
delineated  in  a  Technical  Objectives  and  Plans  (TOSeP)  document; 
that  document,  along  with  the  TO&Ps  for  the  other  projects,  is 
incorporated  into  the  MITRE  contract  with  the  sponsoring  agency. 
The  T08cP  is  signed  by  the  government  program  director  or  project 
officer  and  by  the  MITRE  project  leader. 

With  the  exception  of  managing  its  own  staff  and  other  resources, 
nothing  on  the  program  is  under  the  Corporation’s  direct  control. 


MITRE  can  neither  legally  direct  the  activities  of  government  person¬ 
nel,  nor  direct  the  activities  of  contractor  personnel  in  any  way  that 
affects  the  scope  of  the  contract  that  governs  their  work,  the  associ¬ 
ated  schedule,  or  costs.  In  fact,  only  the  government  contracting 
officer  can  give  such  direction.  MITRE  must  work  with  the  program 
director,  director  of  engineering,  and  integrated  product  team  to 
influence  the  events  on  a  program.  The  Corporation  is  able  to  affect 
the  success  of  ?  program  through  competence,  initiative,  persistence, 
and  reputation.  Most  of  all,  MITRE  is  able  to  influence  the  course  of 
a  program  by  showing  over  time  that  its  advice  is  correct,  and  that 
what  the  Corporation  recommends  is  in  the  best  interest  of  achieving 
the  necessary  system  capability  on  a  reasonable  schedule  and  at 
reasonable  cost. 

As  discussed  in  Chapter  4,  an  ESC  regulation  states  that  MITRE 
is  expected  to  take  the  initiative  in  the  interest  of  achieving  the 
desired  system  capability.  This  means  that  the  Corporation  closely 
watches  all  factors  that  may  affect  the  success  of  the  program — not 
just  the  technical  ones,  but  the  political  and  economic,  as  well. 

When  the  Corporation  recognizes  that  an  action  is  required,  it  is 
incumbent  on  its  project  personnel  to  work  with  the  program  team 
to  ensure  that  appropriate  action  is  initiated  and  completed.  MITRE 
does  not  have  the  role  or  the  resources  to  fill  in  all  the  gaps.  Some 
such  activities  may  be  appropriate  for  MITRE;  most  will  have  to  be 
accomplished  by  other  agencies  and  organizations  associated  with 
the  program. 

As  stated  in  the  introduction  to  this  chapter,  the  environment 
within  which  C^I  system  acquisition  programs  take  place  is  extensive 
and  complicated.  It  is  also  very  dynamic.  Something  unexpected 
arises  every  day  on  such  programs.  Important  people  ask  questions 
and  expect  immediate  answers,  progress  varies  fiom  what  had  been 
anticipated,  a  change  in  some  detail  must  be  made.  The  daily  man¬ 
agement  challenge  presented  to  a  program  director  is  almost  over¬ 
whelming.  Plans  have  to  be  canceled  and  new  ones  made.  Long  hours 
of  work — nights  and  weekends,  and  extensive  travel — are  required. 


A  program  director  needs  to  learn  who  can  be  depended  upon  to  be 
there  when  he  or  she  needs  help. 

When  MITRE  assumes  the  systems  engineering  role,  the  Corpora¬ 
tion  is  dedicated  to  doing  whatever  is  necessary  to  achieve  the  needed 
capability,  and  that  includes  helping  the  program  team  in  any  way 
possible.  Any  >tem  problem  that  the  program  faces  is  a  problem  for 
MITRE.  The  Corporation’s  staff  members  are  prepared  to  make  the 
same  sacrifices  in  time  and  travel  that  are  required  of  the  government 
personnel  to  achieve  the  capability.  MITRE  has  repeatedly  demon¬ 
strated  a  willingness  and  ability  to  go  where  the  action  is  whenever 
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necessary.  On  any  given  project,  when  the  Corporation  acts  in  this 
way,  the  program  director  and  the  other  members  of  the  program 

office  believe  the  Corporation  is  a  full  partner,  ready  to  do  whatever  government's  role,  but  con  eorn 

is  necessary  to  help  them  complete  the  acquisition  program  success- 

0  working  relotionship  in  which 

fully.  MITRE  cannot  assume  the  government’s  role,  but  can  earn  a 

working  relationship  in  which  the  program  director  and  the  rest  of  the  ptogrom  director  ond  the 

the  program  team  know  that  no  matter  what  situation  arises,  the 

^  ,  „  rest  of  the  progrom  teom  know 

Corporation  will  find  a  way  to  help.  MITRE  becomes  a  full  partner 

with  the  program  office  in  carrying  out  the  acquisition  program.  In  pQ  [pofter  whot  situotion 

such  a  role,  MITRE  staff  may  have  a  truly  significant  impact  on 

achieving  the  required  capability.  ° 

While  striving  to  be  a  full  partner  with  the  program  office,  MITRE  yp 

must  act  as  a  professional  organization.  MITRE  was  formed  as  an 
independent  entity  in  part  because  the  government  recognized  a  need 
for  advice  that  was  independent  of  the  government  itself.  The  govern¬ 
ment  wants  MITRE’s  best  advice,  and  the  Corporation  is  dedicated 
to  providing  it.  Occasionally,  there  are  times  when  some  people  in 
government  do  not  want  to  hear  what  MITRE  has  to  say.  People  tend 
not  to  want  to  hear  that  a  program  is  heading  for  trouble,  or  that 
they  should  take  out  some  risk  insurance  in  a  certain  technical  area, 
or  that  more  money  or  time  is  needed,  or  that  action  must  be  taken 
to  modify  the  existing  program  to  achieve  the  capability.  However, 

MITRE  is  paid  to  tell  people  what  they  need  to  hear,  not  just  what 
they  want  to  hear.  Of  course,  MITRE  has  to  act  professionally  in  this 


regard,  recognizing  the  roles  and  responsibilities  of  the  other  program 
participants  and  especially'  those  of  the  program  director  for  whom 
it  works. 

As  noted  above,  MITRE  has  to  be  available  whenever  needed.  At 
the  same  time,  the  Corporation  cannot  provide  “body-shopping”  of 
technical  staff  to  the  government.  MITRE  is  contractually  committed 
to  producing  the  products  described  in  the  TO&Ps,  but  the  Corpora¬ 
tion  also  responds  to  unanticipated  program  needs  as  they  develop. 
Resolution  of  any  conflict  that  arises  is  worked  out  between  counter¬ 
parts,  or — barring  resolution — ultimately  between  the  system  pro¬ 
gram  director,  director  of  engineering,  and  MITRE  project  leader. 

The  MITRE  project  leader  has  two  main  functions.  More  than  any¬ 
one  else,  the  project  leader,  as  the  program  director’s  MITRE  focal 
point,  has  to  be  there  when  needed  and  must  be  fully  cognizant  of  all 
matters  related  to  the  acquisition  program.  The  project  leader  must 
act  in  such  a  way  that  the  program  director  believes  the  Corporation 
will  help  in  any  way  possible.  As  a  second  challenge,  the  project 
leader  must  ensure  that  the  staff  delivers  products  as  required  by  the 
TO&P,  while  at  the  same  time  responding  quickly  to  unanticipated 
program  needs.  The  combination  of  these  two  demands  is  a  signifi¬ 
cant  management  challenge  for  the  MITRE  project  leader. 

In  each  annual  TO&P,  the  program  director  and  MITRE  project 
leader  agree  to  a  series  of  products  that  will  be  produced  by  MITRE 
over  the  course  of  the  coming  year.  These  products  are  explicitly 
identified  and  scheduled.  The  Corporation  is  contractually  bound  to 
meet  those  commitments,  and  the  project  leader  is  responsible  for 
providing  the  products  as  required.  At  the  same  time,  the  project 
leader  must  adjust  the  use  of  the  MITRE  resources  almost  daily  to 
respond  to  the  needs  of  the  program  director.  Some  program  direc¬ 
tors  and  project  leaders  choose  to  write  T08cPs  in  ways  that  explic¬ 
itly  describe  and  limit  the  Corporation’s  commitments.  Others  feel  it 
best  to  write  the  TO&Ps  in  more  general  terms  to  be  free  to  respond 
to  unforeseen  demands.  Whatever  approach  is  taken,  several  things 
must  be  kept  in  mind:  MITRE  does  not  provide  personal  services;  it 


provides  discrete  products.  The  products  to  be  delivered  must  be 
included  in  the  TO&P,  and  the  Corporation  is  legally  respov;sible  tor 
providing  them.  The  government  is  required  to  ensure  that  the  prod¬ 
ucts  are  delivered  and  are  satisfactory.  If  during  the  year  the  program 
director  and  the  project  leader  agree  that  the  planned  product  deliver¬ 
ies  must  be  changed  in  some  way,  then  the  TOdcP  must  be  modified. 
People  sometimes  complain  that  changing  the  TO&P  is  too  time- 
consuming  or  not  worth  the  bother.  But  it  must  be  done.  While  the 
process  is  going  on,  any  direction  given  to  the  Corporation  that 
changes  the  planned  products  should  be  recorded  in  writing  by  the 
MITRE  project  leader  and  copies  made  available  to  the  program 
director.  It  is  important  that  the  proje^.  leader  keep  the  record  of  the 
MITRE  commitments  current  and  accurate.  Changes  in  the  work 
program  should  be  made  as  necessary;  the  need  to  keep  the  documen¬ 
tation  up  to  date  should  not  inhibit  such  changes. 

In  the  ideal  project  leader/program  director  relationship,  the  pro¬ 
gram  director  asks  the  project  leader’s  advice  on  all  important  matters 
related  to  the  system  acquisition  before  taking  action.  The  project 
leader  responds  with  timely,  cogent  advice  that  the  program  director 
can  implement.  To  do  so  requires  MITRE  to  perform  the  technical 
work  necessary  to  provide  a  solid  foundation  for  any  recommenda¬ 
tions.  MITRE’s  advice  is  considered  seriously  and  in  most  cases  fol¬ 
lowed.  When  it  is  not  followed,  the  reasons  are  discussed  and  agreed 
on.  Over  time,  deep  mutual  respect  develops  and  the  program  director 
believes  he  or  she  can  count  on  the  Corporation  to  help  the  program, 
no  matter  what  the  situation.  Clearly,  such  a  relationship  is  earned,  not 
bestowed  upon  MITRE.  On  each  project,  the  MITRE  project  leader  is 
charged  with  earning  such  a  role.  When  it  is  earned,  the  Corporation 
can  have  a  major,  positive  effect  on  the  success  of  the  acquisition 
program  and  on  the  achievement  of  the  desired  capability. 

Obviously,  advice  that  the  program  director  cannot  implement 
does  not  help  to  achieve  the  capability.  Worse  than  that,  it  indicates 
to  the  program  director  and  to  others  that  MITRE  project  personnel 
are  not  appropriately  knowledgeable  of  important  aspects  of  the 
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situation.  Such  occasions  may  also  become  confrontational  when  the 
program  director  does  not  try  to  implement  the  advice.  At  a  mini¬ 
mum,  giving  advice  that  cannot  be  implemented  undermines  MITRE’s 
relationship  with  the  rest  of  the  team;  it  tends  to  make  the  program 
director  uneasy  about  the  MITRE  recommendations  and  may  cause 
the  pro  ;r.im  director  to  double-check  everything  the  Corporation 
says.  A  relationship  in  which  the  program  director  feels  it  necessary 
to  independently  verify  everything  he  or  she  hears  from  MITRE  is 
unsatisfactory,  both  for  the  government  and  for  the  Corporation.  Eor 
the  government,  it  is  costly  and  counterproductive.  Eor  MITRE,  the 
project  staff  will  resent  having  their  advice  constantly  subjected  to  a 
second  opinion.  Carried  to  an  extreme,  such  a  relationship  is  counter¬ 
productive  to  the  achievement  of  the  capability.  MITRE  does  not 
expect  its  advice  to  be  blindly  accepted  and  implemented.  However, 
MITRE  seeks  a  relationship  in  which  it  is  able  to  provide  responsive, 
insightful  advice  that  is  valuable  enough  to  become  the  preferred 
program  office  course  of  action.  This  professional  respect  must  be 
earned.  In  part,  it  is  earned  by  demonstrating  that  the  Corporation 
understands  all  the  circumstances  surrounding  the  situation  at  hand, 
and  factors  them  into  providing  implementable  advice. 

As  obvious  as  the  need  for  implementable  advice  may  seem,  it  is 
important  to  stress  here,  since  the  circumstances  that  surround  pro¬ 
gram  decisions  are  often  very  complicated.  Rarely,  for  example,  is  the 
resolution  of  a  problem,  or  the  seizing  of  an  opportunity,  limited  to 
strictly  technical  considerations.  In  addition,  the  decisions  that  must 
be  made  during  a  major  acquisition  program  have  significant  factors 
that  are  judgmental  in  nature  and  not  amenable  to  strict  measure¬ 
ment.  In  making  such  decisions,  one  is  often  attempting  to  predict 
things  that  may  not  occur  for  many  months,  or  even  years.  It  is  there¬ 
fore  very  difficult  to  achieve  a  relationship  in  which  the  program 
director  has  implicit  faith  in  the  applicability  of  .\li  FRE's  advice. 
Unfortunately,  even  one  case  in  which  the  (.'orporation's  advice  is 
inappropriate  for  the  situation  can  badly  damage  the  MITRE/program 
director  relationship.  To  earn  the  role  of  privy  counselor,  MITRE 


must  fully  understand  all  the  circumstances  under  which  the  program 
director  is  operating  and  make  its  recommendations  accordingly. 
Compromise  among  the  salient  factors  is  often  necessary. 

As  MITRE  demonstrates  a  record  for  good  advice,  there  is  less  need 
for  the  Corporation  to  prove  what  is  being  said  beyond  a  shadow  of  a 
doubt  before  the  program  director  will  take  action.  As  in  everything,  to 
do  something  different  from  what  one  is  currently  doing  takes  time, 
effort,  and  often  resources,  such  as  money.  If  MITRE  suspects  a  prob¬ 
lem,  it  is  only  natural  that  a  program  director  will  want  to  be  sure  of 
the  situation  before  taking  action,  such  as  redirecting  a  contractor.  On 
the  other  hand,  the  longer  one  takes  to  react  to  a  suspected  problem, 
the  more  difficult  and  costly  it  will  be  to  rectify.  The  Corporation  mncr 
avoid  crying  wolf  every  time  a  problem  comes  up.  At  the  same  time,  it 
must  work  to  identify  important  problems — or  opportunities,  for  that 
matter — as  early  as  possible  and  to  take  action  on  them  as  soon  as  the 
evidence  suggests  that  a  change  is  in  order.  There  is  every  reason  for  a 
program  director  to  want  to  delay  dealing  with  a  potential  problem 
until  the  contractor  in  question  owns  up  to  it,  especially  if  the  contract¬ 
ing  situation  is  such  that  the  problem  clearly  lies  with  the  contractor.  If 
the  government  brings  up  the  problem  and  directs  a  change,  the  cost  of 
the  change  may  fall  to  the  government.  If  the  contractor  admits  to  the 
problem  first,  that  contractor  may  be  liable  for  the  associated  costs  of 
the  change.  This  is  a  very  important  consideration,  and  MITRE  must 
weigh  it  heavily.  However,  the  overriding  consideration  should  always 
be  the  best  thing  to  do  under  the  current  circumstances  to  achieve  the 
required  capability  in  a  timely  and  cost-effective  manner.  Sometimes, 
MITRE  must  firmly  and  professionally  urge  the  program  director  to 
take  action  in  the  name  of  the  capability,  even  when  the  action  will 
cost  the  government  money  and  resources  to  take  the  initiative. 

Some  of  the  factors  that  one  may  consider  in  assessing  the  strength 
of  MITRE’s  relationship  with  the  program  director  on  any  given 
project  are  listed  below: 

•  Is  the  MITRE  staff  fully  informed  of  all  program-related 
matters  as  soon  as  they  occur.'  If  the  C.orporation  has  only 
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selective  access  to  information,  or  if  the  staff  hears  about 
events  only  long  after  they  take  p'ace,  the  relationship  is 
not  good,  and  MlTRE’s  ability  to  help  as  systems  engineer 
is  correspondingly  diminished. 

•  Does  MITRE  get  a  chance  to  advise  on  all  important 
program  matters?  Again,  if  not,  the  Corporation  cannot 
perform  as  well  as  it  might  in  the  systems  engineering  role. 

•  Does  MITRE  have  to  prove  everything  it  says  beyond  a 
shadow  of  a  doubt?  Is  the  Corporation’s  advice  accepted 
as  professional  advice?  If  the  first  response  to  any  MITRE 
advice  is  skepticism,  the  relationship  is  poor  and  the 
Corporation’s  impact  will  be  low  and  unnecessarily  costly. 

•  Are  MITRE  people  participants  in  all  key  program  meet¬ 
ings,  especially  those  with  higher  headquarters  personnel 
and  with  user  representatives?  It  is  critical  for  MITRE 
people  to  hear  firsthand  what  other  program  participants 
believe. 

•  Do  MITRE  people  get  a  chance  to  discuss  those  subjects 
that  the  Corporation  understands  best  when  they  are 
presented  to  agencies  other  than  the  program  office?  If 
MITRE  knows  the  material  best,  its  people  can  do  the  best 
job  of  discussing  the  material  with  others.  This  approach 
also  gives  other  agencies  an  appreciation  for  the 
Corporation’s  contribution  to  the  program,  a  question 
often  on  the  minds  of  people  in  Washington  who  have  to 
pay  the  bills. 

None  of  the  above  has  to  be  done  in  a  way  that  makes  it  appear 
that  MITRE  is  appropriating  the  role  of  the  system  program  director. 
If  it  appears  that  MITRE  has  usurped  the  responsibility  of  the  pro¬ 
gram  director,  whether  true  or  not,  the  program  may  be  hurt.  MITRE 
cannot  substitute  for  the  government  in  a  program  direction  role  and 
must  avoid  appearing  to  do  so  while  still  filling  the  legitimate  role  of 
the  systems  engineer.  If  the  project  leader  and  program  director  have 


developed  a  sense  of  mutual  trust  and  respect,  their  joint  efforts  will 
be  perceived  as  cooperative  and  healthy.  Their  achievements  will  far 
exceed  those  of  any  relationship  in  which  MITRE  merely  proposes 
actions,  the  program  director  disposes  as  he  or  she  sees  fit,  and  the 
Corporation  is  content  with  the  result,  regardless  of  whether  the 
advice  was  followed.  MITRE  needs  to  understand  and  accept  situa¬ 
tions  where  its  advice  cannot  be  followed,  or  when  it  is  judged  inap¬ 
propriate  at  the  time  rendered.  However,  when  the  Corporation  is 
giving  good  advice,  such  occasions  should  be  rare. 

Although  MITRE  cannot  assume  the  role  of  the  government  in 
program  direction,  its  personnel  may  be  asked  to  chair  meetings, 
witness  tests,  and  do  other  tasks  that  might  normally  be  done  by 
government  personnel.  However,  the  results  of  such  activities  can  be 
enforced  only  by  government  personnel.  Occasionally,  MITRE  is 
asked  to  take  on  extraordinary  roles  when  government  personnel  are 
unavailable.  Each  occasion  is  carefully  negotiated  to  be  sure  that  the 
Corporation’s  role  is  in  strict  compliance  with  government  regula¬ 
tions.  In  two  cases — the  Strategic  Air  Command  Digital  Information 
Network  (SACDIN)  and  Peace  Sentinel  programs — MITRE  was  asked 
to  assume  the  functions  normally  performed  by  the  program  office 
engineering  divisions. 

For  SACDIN,  the  SPO  did  not  have  an  engineering  division  and 
MITRE  was  asked  to  assume  many  of  the  functions  normally  carried 
out  by  such  a  division.  The  activities  performed  by  MITRE  and  its 
experience  in  that  role  are  described  in  detail  in  a  MITRE  paper.^’ 
The  paper  also  describes  MITRE’s  relationships  to  the  various  direc¬ 
torates  within  the  SACDIN  SPO. 

Peace  Sentinel  was  the  ESC  acquisition  program  that  provided 
Saudi  Arabia  with  its  own  AWACS  aircraft.  MITRE  was  asked  as 
part  of  the  system  engineering  role  to  assume  the  usual  functions  of 
the  SPO  engineering  and  test  directorates.  The  Corporation  was 
directly  involved  in  contractor  negotiations,  a  very  important  acti  1/ 
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that  sometimes  seriously  affects  the  success  of  a  program  and  to 
which  MITRE’s  technical  knowledge  and  operational  understanding 
can  make  important  contributions.  The  Corporation  initiated  techni¬ 
cal  interchange  meetings  with  other  participants  and  often  chaired 
them.  Both  technical  and  programmatic  aspects  of  a  subject  were 
always  considered.  MITRE  became  more  directly  involved  in  coordi¬ 
nation  within  the  program  between  the  SPO  divisions  and  between 
the  SPO  and  other  agencies,  such  as  Air  Force  Systems  Command  and 
the  Air  Staff.  MITRE  was  responsible  for  helping  to  coordinate  its 
technical  contributions  with  the  information  from  the  other  SPO 
divisions,  such  as  program  control,  and  to  provide  the  implementing 
correspondence  for  signature  by  the  system  program  director.  MITRE 
helped  the  SPO  prepare  for  program  reviews  and  prepared  and 
briefed  program-wide  position  papers  and  action  item  status.  This 
expanded  role  involved  the  Corporation  in  much  of  the  great  volume 
of  correspondence  that  typifies  a  SPO  activity,  and  it  consumed 
valuable  MITRE  resources.  This  was  compensated  for  by  a  much 
greater  opportunity  to  contribute  to  a  very  successful  program.  All 
the  Corporation’s  effort  was  conducted  in  an  atmosphere  of  signifi¬ 
cant  U.S.  political  concerns  with  an  associated  international  dimen¬ 
sion.  MIFRE’s  accomplishments  on  the  Peace  Sentinel  Program  were 
recorded  in  an  unnumbered  MITRE  paper. 

To  complete  this  discussion  of  MITRE’s  systems  engineering  role 
in  support  of  a  system  program  director,  two  matters  of  some  sensi¬ 
tivity  deserve  to  be  mentioned.  One  concerns  the  situation  in  which  a 
MITRE  project  leader  and  a  program  director  have  a  difference  of 
opinion  on  a  matter  judged  to  be  important  to  the  success  of  the 
program.  In  such  a  situation,  the  MITRE  project  leader  is  expected  to 
discuss  the  matter  with  his  or  her  supervisor,  who  may  in  turn  discuss 
the  matter  with  the  program  director.  Higher  levels  of  corporate 
management  may  also  become  involved.  In  cases  where  the  situation 
is  deemed  by  MITRE  management  to  be  of  serious  concern  to  the 
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success  of  the  program,  and  where  satisfactory  resolution  cannot  be 
achieved  with  the  program  director,  MITRE  management  will  discuss 
the  situation  with  the  program  director’s  management,  up  to  and 
including  the  Commander  of  ESC,  in  the  case  of  the  Air  Force.  The 
Corporation  will  support  whatever  decision  comes  out  of  that  pro¬ 
cess.  This  process  is  sometimes  the  cause  of  friction  between  MITRE 
and  Air  Force  program  directors.  Everything  possible  is  done  by  the 
Corporation  to  keep  the  program  director  fully  informed  and  in¬ 
volved  in  all  such  discussions.  The  discussions  are  intended  only  to 
ensure  that  when  a  serious  matter  of  major  program  import  arises, 
and  when  MITRE’s  advice  is  not  followed,  that  the  Corporation’s 
contractual  obligation  to  ESC  is  fully  discharged. 

A  second  area  that  is  sometimes  controversial  is  the  interaction  of 
MITRE  people,  especially  senior  management,  with  DOD  and  Air 
Force  personnel  involved  in  some  way  with  an  acquisition  program. 
Such  interactions  can  be  of  great  concern  to  a  program  director  unless 
they  are  handled  very  carefully.  On  the  other  hand,  if  the  relationship 
between  the  program  director  and  the  Corporation  is  healthy,  the 
ability  of  top  corporate  management  to  talk  to  senior  officials  in  the 
DOD  and  the  Air  Force  can  be  of  great  benefit  to  the  program  director 
and  to  the  acquisition  program  itself.  MITRE  management’s  normal 
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responsibilities  bring  them  into  regular  contact  with  such  officials. 
When  a  program  director  wants  to  deliver  a  message  or  gain  some 
necessary  information,  MITRE  management  can  be  an  effective  means 
for  doing  so.  Everyone — the  program  director  and  his  or  her  supervi¬ 
sors,  along  with  MITRE  staff  and  management — needs  to  recognize 
the  value  of  such  a  process  when  well  done,  and  the  potential  for 
serious  problems  if  it  is  not.  The  process  can  work  well  if  MITRE 
people  are  careful  not  to  abuse  it.  The  more  astute  program  directors 
recognize  that  this  MITRE  ability  can  be  helpful  in  accomplishing  tasks 
or  obtaining  information  that  they  might  find  difficult  to  achieve 
without  going  through  many  time-consuming  layers  of  review. 

When  the  desired  MITRE  role  is  achieved,  the  Corporation  is  in  a 
position  to  have  tremendous  influence  on  the  success  of  the  program. 
That  position  places  a  formidable  responsibility  on  MITRE  to  ensure 
that  its  advice  is  indeed  in  the  best  interest  of  achieving  the  required 
capability.  This  role  and  the  associated  responsibility  are  the  chal¬ 
lenge  and  the  satisfaction  enjoyed  by  the  MITRE  staff.  They  are  the 
reasons  that  dedicated  professionals  work  at  MITRE — the  challenge 
of  the  responsibility  and  the  chance  to  do  something  significant 
toward  the  achievement  of  important  national  capabilities. 

MITRE  is  prepared  to  take  extraordinary  steps  to  help  the  govern¬ 
ment  achieve  required  capabilities  and  often  does  so  on  programs  for 
which  it  has  systems  engineering  responsibility.  At  the  same  time, 
both  the  Corporation  and  the  government  must  observe  the  rules  that 
govern  the  use  of  organizations  such  as  MITRE.  The  Corporation 
cannot  usurp  the  roles  of  either  government  or  industry,  cannot 
body-shop  staff,  and  must  studiously  avoid  even  the  appearance  of  a 
conflict  of  interest.  Only  in  these  ways  will  the  special  relationship 
between  the  Corporation  and  the  government  be  preserved  for  future 
important  acquisition  programs. 


mitre's  Relationship  to  Industry 

MITRE  is  paid  by  the  government  to  act  on  the  government’s 
behalf  in  achieving  required  system  capabilities.  However,  neither 
MITRE  nor  the  government  builds  the  systems — profit-making 


industry  does.  Without  industry,  there  are  no  systems.  A  part  of  the 
Corporation’s  role,  then,  involves  helping  the  government  get  what  it 
needs  from  the  industry  that  builds  systems.  In  that  role — when 
MITRE  is  working  effectively — the  Corporation  can  be  as  useful  to 
industry  as  it  is  to  the  government. 

Clearly,  a  confrontational  relationship  between  the  government 
and  industry,  and/or  between  MITRE  and  industry,  is  counterpro¬ 
ductive  to  achieving  the  required  capability.  MITRE  staff  should 
strive  for  a  relationship  of  mutual  respect  and  understanding  with 
industry.  This  is  especially  important  among  the  key  people  in  each 
organization.  Each  group  should  feel  the  other  is  dedicated  to  doing 
its  part  to  achieve  the  required  capability  and  is  making  contribu¬ 
tions  to  providing  it.  Industry  should  feel  that  the  Corporation  exists 
to  help  them,  not  harass  them.  MITRE  should  feel  that  industry  is 
doing  everything  reasonable  to  achieve  the  capability,  not  just  the 
minimum  they  can  get  away  with.  Realizing  such  a  relationship  in 
the  real  world  is  not  easy  to  do,  but  it  is  an  important  ingredient  of  a 
successful  program.  It  is  fostered  by  frequent,  open  discussion  at  all 
levels  and  by  offering  constructive  suggestions,  rather  than  just 
criticism.  Mutual  respect  between  the  Corporation  and  industry 
grows  or  erodes,  as  the  case  may  be,  as  the  program  passes  through 
the  various  phases  of  acquisition. 

Because  of  MITRE’s  role  between  industry  and  government,  the 
Corporation  is  often  placed  in  situations  in  which  it  evaluates  the 
products  of  industry  against  the  government’s  requirements,  or  in 
which  it  assesses  the  progress  of  industry  in  accomplishing  its  con¬ 
tractual  commitments.  MITRE  staff  must  act  as  an  honest  broker  and 
must  constantly  ask,  “What  is  the  best  thing  to  do,  under  the  circum¬ 
stances  that  prevail  at  this  time,  to  ensure  the  required  capability  is 
achieved  in  a  reasonable  time  and  at  a  reasonable  cost?”  Often,  the 
answer  to  that  question  will  require  both  government  and  industry 
action  and  may  meet  resistance  on  both  sides.  In  this  process,  one 
needs  to  separate  carefully  what  is  best  to  do  at  this  time  from  how 
the  circumstances  developed  and  who  should  reimburse  whom  in 


view  of  the  required  changes.  Forcing  contract  compliance  may  not 
be  the  best  action.  The  existing  contract  is  an  important  factor,  but 
not  the  only  one.  Once  agreement  is  reached  on  what  needs  to  be 
done,  fault  and  liability  can  be  adjudicated.  Resolution  cannot  be 
achieved  the  other  way  around,  although  many  will  wish  to  talk  more 
about  liability  than  capability. 

MITRE’s  relationship  with  a  particular  contractor  on  a  given 

program  is  in  part  established  by  what  has  taken  place  on  earlier 

programs  in  which  the  two  have  worked  with  one  another.  However, 

i  the  relationship  results  mainly  from  the  interactions  that  take  place 

on  the  current  program.  There  are  always  many  opportunities  for 

Forcing  contract  compliance  .  rr 

MITRE  to  make  positive  contributions  to  industry  efforts, 
may  not  be  the  best  action.  The  process  used  by  the  government  to  select  a  contractor  to  build  a 

system  involves  competition  among  industrial  companies  able  to  do  the 

The  existing  contract  is  an 

job.  This  process  is  known  as  source  selection.  The  competition  is  based 
importont  factor,  but  not  on  who  can  do  the  job,  and  for  what  price,  as  represented  by  industry 
proposals.  This  approach  forces  a  contractor  to  propose  as  much  capa- 
e  only  one.  bility  as  possible,  for  as  low  a  price  as  possible,  to  win  the  competition. 

Whether  some  alternative  approach  would  result  in  better  systems  is  not 
a  subject  of  this  book.  During  source  selection,  MITRE  helps  the  gov¬ 
ernment  narrow  down  the  proposals  to  those  that  are  most  responsive 
to  the  government’s  requirements,  and  then  to  choose  the  best  from  that 
list.  The  Corporation  does  not  have  a  vote  on  which  company  should  be 
selected  but  provides  technical  evaluations  of  contractor  proposals  to 
those  government  people  who  do.  A  contract  should  not  be  awarded 
when  the  government  believes  that  the  program  cannot  be  accomplished 
for  the  time  and  money  proposed.  This  is  a  critical  concern.  Both  indus¬ 
try  and  government  must  believe  that  the  job  can  be  done  within  the 
available  time  and  money.  MITRE  must  be  prepared  to  attest  to  that 
belief  at  the  time  of  contract  award,  or  to  recommend  alternative 
courses  of  action  to  provide  consistency  among  the  required  capabilities, 
money,  and  time  provided  in  the  government/industry  contracts.  A 
program  that  is  initiated  on  any  other  basis  is  certain  to  have  major 
problems  during  acquisition. 


mitre’s  capabilities  are  clearly  useful  in  assessing  the  technical 
quality  and  feasibility  of  industry  proposals  submitted  in  response  to 
a  government  RFP  to  build  a  new  system.  With  strong  professional 
people  from  MITRE  involved,  corporations  may  expect  that  the 
technical  strength  of  their  proposals  will  be  recognized  and  will  be  a 
major  factor  in  the  award  of  the  contract.  MlTRE’s  knowledge  of 
what  it  takes  to  build  certain  capabilities  helps  the  government  to 
estimate  the  realism  of  contractors’  price  and  schedule  proposals  and 
to  differentiate  fairly  among  the  proposers. 

With  a  proper  foundation,  all  participants  can  assume  a  good-faith 
attitude  as  the  program  begins.  In  particular,  the  contractor  will  begin 
the  program  with  the  approach  that  it  fully  intends  to  deliver  what  is 
required  by  the  contract,  within  the  time  and  resources  provided. 
MlTRE’s  attitude  about  the  contractor  must  reflect  the  belief  that  the 
contractor  is  so  committed.  The  Corporation  is  not  in  the  business  of 
trying  to  manipulate  the  situation  to  extract  the  last  ounce  of  capabil¬ 
ity  from  the  contractor,  whether  the  contract  provides  for  it  or  not. 
MITRE  is  not  driven  by  the  belief  that  contractors  are  out  to  exploit 
the  government  for  corporate  gain.  At  the  same  time,  MITRE  sta'^f 
cannot  be  so  naive  as  to  believe  that  no  matter  what  happens,  the 
contractor  will  deliver  as  required  by  the  terms  of  the  contract. 

As  the  program  proceeds,  MlTRE’s  technical  skills,  experience  with 
related  mission  capabilities,  and  understanding  of  the  government’s 
requirements  can  be  helpful  to  the  industry  in  meeting  its  contractual 
commitments.  MITRE  can  help  industry  translate  government  require¬ 
ments  into  industry  technical  specifications  to  be  used  as  the  basis  for 
constructing  the  system.  Interpretations  can  be  provided  and  questions 
answered.  One  MITRE  activity  that  has  proved  successful  is  v,  rking 
directly  with  the  contractor  early  in  the  program,  sometimes  at  the 
contractor’s  plant,  to  clarify  any  questionable  areas  in  the  govern¬ 
ment’s  specification.  In  that  way,  the  contractor  work  will  get  started 
in  the  right  direction  and  the  need  for  later  redirection  will  be  mini¬ 
mized.  Mutual  respect  and  confidence  will  be  built.  Such  dialog  should 
continue  throughout  the  program.  As  discussed  below,  both  MITRE 


As  the  program  proceeds, 
MITRE'S  fechnicol  skills, 
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capabilities,  and  understonding 
of  the  government's 
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the  industry  in  meeting  its 
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When  MITRE  observes  that  the 
contractor  is  not  fulfilling  the 
requirements  of  the  contract,  or 
thot  the  controctor  design  hos 
flows,  or  that  the  tests  hove 
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openly,  and  professionally. 


and  the  contractor  must  recognize  that  MITRE  cannot  legally  direct 
the  contractor  in  any  way  that  affects  the  performance,  schedule,  or 
cost  aspects  of  the  government/industry  contract.  Despite  that,  the 
Corporation’s  advice  and  counsel  can  be  helpful  to  the  contractor  in 
meeting  the  terms  of  its  contract. 

Conflicts  may  arise  in  trying  to  help  both  government  and  industry 
achieve  the  required  capability.  These  conflicts  may  involve  questions 
of  whether  the  contractor  has  done  what  was  promised,  or  whether 
the  government  has  changed  its  requirements,  or  even  whether  the 
job  is  still  doable  or  desirable.  Helping  to  resolve  those  conflicts  in 
the  best  interests  of  attaining  the  required  mission  capability  is  a 
major  challenge  to  MITRE  staff.  The  Corporation’s  technical  capabil¬ 
ity  and  familiarity  with  the  details  of  the  contractor’s  work  can  be  of 
great  value  to  industry,  as  well  as  to  the  government,  in  such  situa¬ 
tions.  MITRE’s  capability  on  the  government  side  provides  a  realistic 
appreciation  for  the  contractor’s  technical  situation.  The  government 
is  not  in  a  position  where  its  only  option  is  to  insist  that  the  terms  of 
the  contract  be  met.  Alternative  actions  can  be  evaluated  from  opera¬ 
tional,  technical,  and  contractual  points  of  view.  Because  the  detailed 
specifics  of  the  situation  arc  better  understood  on  the  government 
side,  the  treatment  of  industry  can  be  more  equitable  than  it  would 
be  otherwise,  and  the  bitter  discussions  that  sometimes  accompany 
adjudication  of  fault  for  program  failures  can  be  minimized. 

It  is  also  clear  that  MITRE’s  technical  knowledge  can  raise  difficult 
questions  for  industry.  However,  in  the  long  run,  achievement  of  the 
required  capability  will  depend  on  good  quality  technical  work  by  the 
contractor.  The  sooner  a  problem  area  is  identified  and  any  necessary 
changes  implemented,  the  more  likely  the  contractor  will  provide  the 
required  capability,  an  objective  in  which  the  contractor  shares.  When 
MITRE  observes  that  the  contractor  is  not  fulfilling  the  requirements 
of  the  contract,  or  that  the  contractor’s  design  has  flaws,  or  that  the 
tests  have  failed,  or  whatever,  the  Corporation  has  a  responsibility  to 
say  so,  clearly,  openly,  and  professionally.  Suggestions  for  fixing  the 
problem  should  also  be  offered.  In  some  sense,  then,  MITRE  represents 


a  threat  to  the  contractor,  a  government  capability  that  can  cause 
trouble  for  the  contractor,  trouble  that  might  not  arise  if  the  Corpora¬ 
tion  were  not  working  on  the  program. 

Thorough  review  of  contractor  efforts  is  one  of  the  major  thrusts 
of  systems  engineering.  MITRE  should  direct  itself  to  those  issues 
that  really  make  a  difference  in  whether  the  capability  is  achieved. 
Harassing  the  contractor  on  every  little  point  is  counterproductive 
to  achieving  the  capability,  since  it  provides  no  improvements, 
creates  animosity,  and  wastes  resources  all  around.  If  the  Corpora¬ 
tion  has  earned  the  respect  of  the  contractor,  acts  professionally, 
and  restricts  its  considerations  to  those  that  in  some  significant  way 
affect  achieving  the  capability,  it  will  be  able  to  critique  the  contrac¬ 
tor  effort  and  have  a  positive  impact.  MITRE  is  not  in  the  business 
of  criticizing  contractors.  Quite  to  the  contrary,  MITRE  is  in  the 
business  of  helping  contractors  build  the  required  systems. 

When  MITRE  and  contractor  personnel  have  a  difference  of  opin¬ 
ion  on  a  technical  matter,  the  Corporation’s  staff  must  be  careful  to 
include  all  relevant  factors  in  coming  to  a  recommended  action,  not 
just  those  of  a  strictly  technical  nature.  MITRE  technical  people  must 
include  in  their  assessment  the  real-world  practicality  of  the  contrac¬ 
tor  taking  MITRE’s  approach,  fully  considering  the  contractor’s 
obligations,  what  has  transpired  on  the  program  to  this  point,  and 
the  best  interests  of  achieving  the  required  capability.  MITRE  must 
avoid  the  “ivory  tower”  approach  that  industry  sometimes  feels 
characterizes  its  recommendations.  MITRE’s  staff  must  be  very 
knowledgeable  about  all  factors,  and  equally  pragmatic.  Over  60 
percent  of  MITRE  staff  is  hired  directly  from  industry,  and  an  even 
larger  percentage  has  had  industrial  experience.  That  experience 
should  be  reflected  in  the  Corporation’s  recommendations. 

In  every  acquisition  program,  the  contractor  is  required  to  provide 
a  significant  amount  of  documentation  for  government  review  and 
approval — specifications,  test  plans,  test  procedures,  etc.  Preparation 
and  review  of  this  documentation  is  a  very  costly  and  tedious  process 
that  can  be  detrimental  to  progress  if  not  handled  carefully.  An 


Contractor  documentation  is  of 
interest  only  to  the  extent  that 
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approach  in  which  industry  prepares  its  documents  without  in-pro¬ 
cess  interaction  with  the  government  and  MITRE,  hands  the  do<_'u- 
ments  over  to  the  government,  and  receives  comments  back,  is  a 
prescription  for  failure.  The  process  will  take  several  itc  '  uions  before 
both  sides  can  agree  on  an  approved  document;  in  the  meantime, 
resources  will  be  wasted  on  both  sides  and  animosity  will  build. 
MITRE  and  the  government  must  work  closely  with  the  industry  as 
the  documentation  is  produced.  Industry  should  feel  free  to  ask  ques¬ 
tions  and  should  get  timely,  definitive  answers.  Comments  and  re¬ 
writes  should  be  prepared  more  as  a  team,  and  less  as  a  “we  versus 
them”  contest.  MITRE  staff  should  expect  to  spend  time  explaining 
their  comments  and  making  suggestions  for  improvements. 

Contractor  dcKumentation  is  of  interest  only  to  the  extent  that  it 
contributes  to  achieving  capability.  Each  required  document  should 
have  a  purpose  and  should  be  evaluated  accordingly.  Purposes  may 
include  describing  what  the  contractor  plans  to  do  for  the  government; 
describing  what  the  contractor  has  done;  or  permitting  someone  other 
than  the  contractor  to  operate  and  maintain  the  system.  Each  docu¬ 
ment  has  to  be  adequate  to  serve  its  purpose — adequate,  nor  of  Pulitzer 
Prize  quality,  not  ideal,  not  even  excellent.  To  repeat  an  admonition 
made  elsewhere  in  this  book,  the  Corporation’s  cominents  should 
distinguish  those  that  are  truly  important  to  achieving  the  capability 
from  those  that  are  merely  to  make  the  document  neat.  MITRE  should 
work  directly  with  the  contractor  in  resolving  the  important  ones  and 
place  significantly  less  emphasis  on  the  remainder.  Clearly,  however,  it 
is  the  contractor’s  responsibility  to  ensure  that  both  the  technical  and 
editorial  quality  of  its  documentation  are  adequate  to  support  success¬ 
ful  system  development  and  implementation. 

Another  facet  of  MlTRE/industry  relationships  involves  the  study 
of  potential  program  actions  or  changes.  There  will  be  many  times 
when  someone  will  propose  an  add,  ion  or  deletion  to  the  system 
being  built,  or  will  propose  some  change  in  the  way  the  system  is 
designed  or  built.  These  proposals  will  generate  many  what  if” 
studies,  studies  that  for  the  most  part  will  not  have  been  anticipated 


when  the  program  began.  MITRE  must  be  very  careful  in  recom¬ 
mending  that  industry  undertake  such  studies.  Industry  may  have  to 
be  involved,  since  it  may  well  have  the  information  necessary  to  make 
a  meaningful  assessment  of  the  proposal,  and  since  the  results  may 
affect  the  work  program.  Such  studies  obviously  consume  industry 
resources  that  were  allocated  to  the  existing  program.  But  more  than 
that,  whether  the  studies  are  conducted  by  industry,  MITRE,  or  some 
other  group,  they  raise  an  uncertainty  in  all  parties  about  how  to 
proceed  pending  their  outcome  and  the  resulting  government  deci¬ 
sions.  This  uncertainty  can  be  very  costly  as  people  delay,  awaiting 
the  outcome.  Some  such  excursions  are  absolutely  necessary  and 
require  industry  participation.  However,  too  much  of  this  sort  of 
thing  undermines  the  resolution  of  the  program  participants  and  can 
result  in  large  costs  to  the  program  and  significant  delays  in  achieving 
the  required  capability.  A  key  part  of  the  MITRE/industry  relation¬ 
ship  involves  the  proper  participation  of  both  groups  in  the  study  of 
alternative  courses  of  action. 

The  inevitability  of  change  on  a  major  acquisition  program  is 
emphasized  throughout  this  book.  Many  of  these  changes  will  impact 
on  the  contractor’s  activity.  Some  will  require  modification  of  the 
contract,  and  some  will  be  caused  by  the  contractor’s  actions  or  lack 
thereof,  or  by  factors  outside  the  contractor’s  control.  As  a  group, 
MITRE  project  personnel  must  be  well  enough  informed  to  recognize 
the  necessity  for  change  and  the  reasons  for  it.  They  must  understand 
the  potential  impact  on  the  resulting  capability  if  change  does  not 
take  place,  or  of  alternative  changes  that  might  be  made.  In  particu¬ 
lar,  with  respect  to  industry,  the  Corporation  must  be  able  to  differ¬ 
entiate  those  changes  that  should  be  handled  within  the  existing 
contract  from  those  that  require  contract  modifications.  As  noted 
above,  MITRE  staff  cannot  naively  assume  that  the  contractor  will 
do  whatever  is  necessary  to  achieve  the  capability.  Each  of  the  major 
players — the  government,  industry  and  MITRE — has  limited  re¬ 
sources,  is  concerned  about  doing  its  part,  and  wants  to  maintain  a 
good  reputation.  Industry  does  not  have  unlimited  funds  and  is 
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properly  and  understandably  not  willing  to  assume  responsibility  for 
events  outside  its  contractual  provisions.  MITRE  simply  cannot 
afford  to  assume  that  somehow  the  industry  will  take  care  of  every 
problem  that  arises.  The  Corporation  should  make  it  clear  when 
industry  should  do  so;  when  additional  resources  are  appropriate,  it 
should  counsel  the  government  to  provide  them. 

Another  temptation  faced  by  MITRE  staff  when  a  problem 
occurs  is  to  pitch  in  and  help  write  the  contractor  test  plan,  or  help 
the  contractor  produce  some  other  product  for  which  it  is  respon¬ 
sible.  The  staff  can  get  frustrated  at  telling  the  contractor  what 
needs  to  be  done  and  then  not  having  it  done.  They  may  conclude 
it  would  be  less  effort  to  do  it  themselves.  On  very  rare  occasions, 
that  is  the  right  course  of  action.  But  there  are  two  words  of  warn¬ 
ing;  First,  the  contractor  is  responsible  for  those  products,  and  if 
MITRE  usurps  that  responsibility,  the  Corporation  is  in  some  sense 
liable  for  the  results.  Even  though  the  Corporation  cannot  legally 
direct  the  contractor,  MITRE  initiatives  such  as  this  are  the  basis 
for  subsequent  arguments  about  who  is  at  fault  when  things  go 
wrong.  Second,  the  government  cannot  afford  to  pay  both  organi¬ 
zations  to  do  the  same  job.  With  limited  resources,  MITRE  would 
have  to  detract  from  another  of  its  responsibilities  on  the  program 
to  assume  one  of  industry’s  tasks.  Therefore,  such  activity  must  be 
negotiated  with  the  system  program  director  and  arranged  with  the 
contractor.  Despite  these  concerns,  on  occasion,  the  best  interests 
of  achieving  the  capability  dictate  such  initiatives.  They  should  be 
undertaken  only  with  the  full  understanding  and  support  of  the 
system  program  director. 

Although  not  as  prevalent  today  as  it  was  years  ago,  there  are 
often  references  to  MITRE  giving  technical  direction  to  the  contrac¬ 
tors  on  programs  for  which  the  Corporation  has  systems  engineering 
responsibility.  It  should  be  noted  that  only  the  contracting  officer  can 
give  the  contractor  direction  that  changes  the  contract’s  performance 
requirements,  schedules,  or  costs.  MITRE  cannot  give  such  direction; 
MITRE  staff  and  contractor  personnel  need  to  understand  that.  The 


contractor  cannot  use  a  MITRE  statement  as  a  basis  for  doing  some¬ 
thing  different  from  what  is  required  by  the  contract.  MITRE  staff 
cannot  expect  them  to  and  should  not  attempt  to  influence  the  con¬ 
tractor  in  that  way.  On  the  other  hand,  the  Corporation’s  activities 
can  have  great  impact  on  what  the  contractor  does.  The  government 
specification  prepared  by  MITRE  helps  to  direct  the  contractor’s 
activities,  as  do  conversations  in  which  the  Corporation  provides 
interpretations  of  those  specifications.  The  specification  is  contractu¬ 
ally  binding  on  the  contractor,  but  its  interpretations  are  not.  MITRE 
comments  on  contractor  documentation,  contractor  progress,  or  test 
results  are  all  a  form  of  direction  when  provided  through  the  SPO, 
but  become  contractually  directive  only  when  the  contracting  officer 
makes  them  so.  The  Corporation  can  have  significant  impact  on  the 
technical  direction  of  the  contractor’s  activities,  but  only  the  contract¬ 
ing  officer  can  give  technical  directiop. 

There  are  other  significant  ways  in  which  MITRE  interacts  with 
industry  outside  individual  programs.  Key  MITRE  people  meet  their 
counterparts  in  industry  to  discuss  common  interests  and  corporate 
relationships.  Both  MITRE  and  industry  people  serve  on  various 
government  committees  and  through  these  associations  build  mutual 
respect.  The  Corporation  regularly  exchanges  information  with  its 
industrial  counterparts  concerning  technology  developments.  In  this 
way,  the  top  technical  people  get  to  know  and  respect  one  another. 
MITRE  may  suggest  new  areas  that  are  of  interest  for  future  govern¬ 
ment  systems  and  represent  new  opportunities  for  industry.  Such 
information  is  useful  to  industry  in  its  determinations  of  where  to 
invest  resources  in  anticipation  of  future  business.  Industry  may 
describe  new  technology  that  is  becoming  available  for  use  by  the 
government.  The  knowledge  of  new  technology  gained  by  the  Corpo¬ 
ration  is  helpful  in  assisting  the  government  in  establishing  new 
acquisition  programs  designed  to  fill  critical  government  require¬ 
ments.  Many  of  the  MITRE/industry  discussions  include  industry 
proprietary  data.  MITRE  procedures  require  that  proprietary  data 
be  protected  as  carefully  as  classified  information. 


The  history  of  ocquisition  pro¬ 
grams  is  replete  with  examples 
of  contractor/subcontractor 
problems.  It  is  an  impartant 
area  of  MITRE  concern  as 
systems  engineer. 


This  section  has  discussed  MITRE’s  interactions  with  the  major 
industrial  corporations  with  whom  the  government  has  a  contract  for 
delivering  a  system.  They  are  the  prime  contractors.  However,  there 
are  a  large  number  of  subcontractors  that  provide  portions  of  the 
system  to  the  prime  contractor,  who  in  turn  delivers  the  system  to  the 
government.  The  contributions  of  the  subcontractors  are  critical  to 
achievement  of  the  capability,  but  subcontractors  work  for  the  prime 
contractors;  they  are  not  under  contract  to  the  government.  Many 
major  failures  on  acquisition  programs  have  resulted  from  problems 
between  prime  and  subcontractors.  It  is  therefore  very  important  that 
MITRE  be  cognizant  of  the  prime/subcontractor  relationships  at  the 
time  of  contract.  The  Corporation  must  monitor  events  in  that  area 
throughout  the  program.  Important  issues  between  the  prime  and  the 
subcontractors  that  affect  achieving  the  capability  must  be  raised  by 
the  Corporation  in  a  timely  manner.  This  may  not  be  easy  to  do.  It 
will  often  be  difficult  to  get  the  required  information;  the  prime 
contractor  may  feel  its  dealings  with  subcontractors  are  its  business 
and  not  the  government's.  However,  MITRE  must  take  the  initiative 
in  the  name  of  achieving  the  capability,  and  the  quality  of  the 
MlTRE/industry  relationship  will  determine  how  effective  the  Corpo¬ 
ration  can  be.  When  the  industry  believes  MITRE  is  both  knowledge¬ 
able  and  acting  in  the  best  interest  of  industry,  it  will  cooperate  for 
the  good  of  the  program.  Again,  the  relationship  must  be  earned  by 
MITRE.  In  any  case,  the  history  of  acquisition  programs  is  replete 
with  examples  of  contractor/subcontractor  problems.  It  is  an  impor¬ 
tant  area  of  MITRE  concern  as  systems  engineer. 

There  is  one  other  segment  of  industry  with  which  MITRE  may 
interact  when  assigned  the  systems  engineering  role  on  a  CM  system 
acquisition  program.  When  the  government  requires  assistance  that  is 
inappropriate  for  MITRE,  or  when  the  Corporation  cannot  provide  the 
required  resources,  the  government  may  hire  another  contractor  to 
provide  that  assistance.  This  is  not  a  new  or  unusual  situation.  When 
MITRE  was  unable  to  provide  all  the  assistance  required  in  the  area 
of  personnel  subsystems  on  the  SAGE  system  in  the  late  1950s,  the 


government  hired  a  contractor  to  provide  analysis  and  documentation 
support.  MITRE  worked  with  that  contractor  to  ensure  the  applicabil¬ 
ity  of  their  products.  More  recently,  the  number  of  C^l  systems  that  the 
government  is  attempting  to  acquire  has  increased  more  rapidly  than 
the  in-house  government  resources  available  to  manage  the  programs. 
This  has  resulted  in  a  significant  growth  in  the  number  of  tasks  the 
government  has  had  to  contract  with  organizations  other  than  MITRE 
or  the  companies  building  the  systems.  These  contractors  are  known  as 
System  Engineering  Technical  Assistance  (SETAs)  or  Technical  Engi¬ 
neering  Management  Services  (TEMS)  corporations.  In  many  cases, 
there  is  little  overlap  between  the  work  of  these  companies  and 
MITRE’s  work  as  systems  engineer,  but  overlaps  sometimes  do  occur. 
For  example,  a  separate  contractor  may  have  responsibility  for  inde¬ 
pendent  verification  and  validation  (IV&V)  of  software  supplied  by  the 
system  contractor.  MITRE  too  is  concerned  with  the  adequacy  of  that 
software.  To  help  manage  that  overlap,  the  Corporation  remains 
cognizant  of  the  IV&V  contractor’s  work  and  incorporates  the  results 
into  the  Corporation’s  recommendations  to  the  SPO. 

In  summary,  MITRE  can  achieve  a  cooperative  and  constructive 
relationship  with  industry  without  appearing  to  take  over  the 
industry’s  responsibilities  and  without  abrogating  its  responsibilities 
to  act  in  behalf  of  the  government.  Again,  mutual  respect  and  trust 
among  key  people  in  government,  industry,  and  the  Corporation  are 
essential  ingredients.  A  relationship  of  this  sort  must  be  earned  by 
MITRE  on  every  program  for  which  the  Corporation  has  systems 
engineering  responsibility. 

The  Acqvisitioii  EnviroinNent 

MITRE  is  responsible  for  achieving  a  thorough,  realistic  under¬ 
standing  and  appreciation  for  the  mission  capabilities  that  the  user 
cominaiids  are  attempting  to  achieve.  As  a  second  major  challenge, 
the  Corporation  must  be  fully  aware  of  the  total  environment  in 
which  every  system  acquisition  takes  place.  For  military  systems,  that 
environment  was  characterized  by  A.D.  Hall  in  1962: 


In  the  systems  engineering  role, 
MITRE  must  thoroughly 
understand  the  universe  in 
which  the  system  is  being 
created  and  must  be  willing 
and  able  to  adapt  to 
that  environment. 


But  let  the  management  be  the  United  States  government  trying 
to  provide  the  military  security  of  the  nation,  and  let  the 
functions  of  research,  systems  engineering,  development,  manu¬ 
facture,  and  operation  be  dispersed  among  hundreds  of  organi¬ 
zations — then  these  objectives  are  approached  only  with  the 
greatest  of  difficulty. 

A  similar  description  of  the  government  acquisition  process  was 
given  by  the  former  commander  of  the  Air  Force  Systems  Command, 
General  A.  Slay  (U.S.  Air  Force  retired)  in  his  testimony  to  the  Senate 
Appropriations  Committee  in  1981: 

We  now  face  a  situation  in  which  at  least  seven  organizational 
layers  or  echelons  do  the  same  jobs. ..and,  far  from  paying  their 
way,  these  multiple  layers  significantly  reduce  the  effectiveness 
with  which  the  overall  systems  development  and  acquisition 
mission  is  performed. 

Today,  most  important  decisions  on  Air  Force  acquisition  pro¬ 
grams  are  made  remote  from  the  Program  Office,  usually  by 
people  who  have  never  been  a  Program  Manager,  have  never 
negotiated  a  contract,  who  have  never  used  an  Air  Force  weapon 
system  in  combat,  and  who  have  no  technical  or  management 
expertise  on  the  program  in  review.'*’^ 

In  his  latest  book  on  systems  engineering,  Hall  defined  a  system 
and  its  environment  as  a  “universe”  or  a  “supersystem.”'*'  In  the 
systems  engineering  role,  MITRE  must  thoroughly  understand  the 
universe  in  which  the  system  is  being  created  and  must  be  willing 
and  able  to  adapt  to  that  environment. 

MITRE  has  no  direct  control  over  any  portion  of  this  environ¬ 
ment.  On  the  other  hand,  the  more  MITRE  understands  of  the  envi¬ 
ronment,  the  more  likely  its  advice  will  be  timely  and  effective.  The 
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more  MITRE  is  able  to  help  the  program  director  accommodate  that 
universe  in  conducting  the  acquisition  program,  the  greater  the  prob¬ 
ability  that  a  timely,  cost-effective  mission  capability  will  be  achieved. 
It  is  also  true  that  the  program  director  has  little  direct  control  over 
many  of  the  factors  that  affect  his  or  her  program.  MITRE  can  be  of 
real  assistance  to  the  program  director  by  being  aware  of  the  environ¬ 
ment  surrounding  the  program  and  by  providing  timely  advice  on 
how  to  deal  with  it  in  a  proactive  manner. 

Before  describing  some  specific  aspects  of  the  environment  one 
finds  in  most  acquisition  programs,  a  few  general  characteristics  are 
worth  mentioning.  The  environment  is  very  large,  complex,  and 
extremely  dynamic.  One  never  knows  from  day  to  day  who  among  all 
the  players  will  ask  an  important  question,  make  a  declaration,  or 
announce  some  finding  or  decision  that  impacts  the  program  in  an 
important  way.  Significant  resources  of  all  parties  are  consumed  in 
responding  to  these  situations.  Many  of  them  are  not  productive,  but 
often  they  cannot  be  ignored.  Some  of  them  will  occur  repeatedly  on 
a  program.  For  example,  the  ability  of  a  system  to  perform  in  a 
countermeasures  environment  is  a  debate  that  goes  on  throughout  the 
program  as  the  system  evolves,  the  enemy  threat  increases,  people 
invent  new  scenarios  to  challenge  the  system,  special  review  groups 
examine  the  system,  and  new  people  assume  positions  that  provide 
some  relationship  to  the  program.  As  a  full  partner  with  the  program 
director,  an  important  part  of  MITRE’s  systems  engineering  work  is 
to  be  cognizant  of  this  environment  at  all  times  and  prepared  to  help 
deal  with  it  in  all  its  dimensions.  Most  especially,  the  Corporation 
must  be  prepared  to  handle  the  most  difficult  technical  questions  in 
any  arena  in  which  they  come  up. 

The  program  director  works  in  a  complex  system  universe.  First, 
there  is  always  an  existing  mission  capability  on  which  the  new  one 
will  be  imposed.  Therefore,  the  planned  system  will  always  be  a 
mixture  of  new  and  existing  elements.  To  be  successful,  the  new 
capability  must  provide  for  evolution  from  the  existing  one.  Similarly, 
the  new  capability  will  evolve  over  many  years,  and  new  features  will 


MITRE  hos  a  unique  opportunity 
and  therefore  responsibility  to 
help  occommodote  interservice 
and  internotionol  considerotions 
in  the  acquisition  program. 


be  added  as  they  become  available.  Lack  of  resources,  technology 
limitations,  and  the  rate  at  which  the  user  can  absorb  new  capabili¬ 
ties  may  all  require  a  phased  implementation. 

As  described  in  Chapter  2,  the  current  DOD  acquisition  process 
develops  individual  subsystems,  or  pieces  of  a  mission  capability. 
Each  subsystem  is  bounded  in  function,  time,  and  money.  No 
provision  is  made  in  the  individual  programs  for  combining  all  the 
subsystems  into  a  mission  capability.  That  task  is  left  ?.s  an  exer¬ 
cise  for  the  using  command.  MITRE  has  a  capability  orientation 
and  is  involved  in  the  acquisition  of  many  of  the  subsystems  that 
constitute  the  capability.  While  helping  the  government  in  provid¬ 
ing  the  separate  subsystems  that  go  to  make  up  the  mission  capa¬ 
bility,  the  Corporation  has  both  a  special  knowledge  and  a  special 
responsibility  to  help  in  achieving  that  capability.  This  approach 
helps  the  Corporation  advise  the  program  director  on  actions  to 
ensure  the  subsystem  being  acquired  integrates  well  with  the  other 
existing  and  planned  pieces  of  the  capability.  This  approach  ex¬ 
pands  the  universe  extensively.  It  is  also  crucial  to  achieving  effec¬ 
tive  mission  systems. 

In  programs  where  interoperation  with  other  services  is  involved, 
or  where  operations  with  the  forces  of  other  nations  are  required,  the 
considerations  of  how  the  new  capability  fits  with  existing  ones  are 
made  immensely  more  difficult  to  understand  and  accommodate  in 
the  program.  To  be  cognizant  of  these  factors  and  see  that  they  are 
factored  into  the  current  program  is  one  of  MlTRE’s  major  system 
engineering  challenges.  On  the  other  hand,  MITRE  has  worked  on 
many  different  programs  with  all  the  U.S.  military  services,  and  on  a 
large  number  of  systems  that  the  U.S.  has  helped  to  acquire  for  other 
friendly  nations.  As  a  result,  the  Corporation  is  uniquely  knowledge¬ 
able  about  programs  that  involve  interoperation  among  U.S.  services 
or  with  U.S.  allies.  It  is  important  to  place  the  current  system  in  its 
proper  universe.  Again,  MITRE  has  a  unique  opportunity  and  there¬ 
fore  responsibility  to  help  accommodate  interservice  and  international 
considerations  in  the  acquisition  program. 


To  make  matters  in  this  regard  even  more  challenging,  the  system  that 
is  being  built  will  most  likely  have  to  interact  with  other  systems  cur¬ 
rently  in  acquisition  or  planned  for  future  acquisition.  Again,  these 
programs  can  have  major  impact  on  the  system  being  acquired.  The 
capabilities  to  be  provided  by  these  other  programs  must  be  appreciated 
by  MITRE  as  systems  engineer  and  their  real  and  potential  impact  built 
into  the  acquisition  program.  This  challenge  is  made  more  difficult  by  the 
tendency  for  the  government  to  buy  subsystems  rather  than  capabilities, 
as  described  above.  The  interactions  between  the  subsystems — the  inter¬ 
faces — tend  to  get  inadequate  attention.  When  one  recognizes  an  inter¬ 
face  problem,  it  is  difficult  to  convince  any  program  director  to  take  the 
initiative  to  provide  for  the  interface.  Quite  naturally,  with  bounded 
direction  and  often  with  very  limited  resources,  a  program  director 
concentrates  those  resources  on  the  system  for  which  he  or  she  has 
explicit  direction. 

On  the  other  hand,  MITRE’s  preoccupation  with  the  overall  capa¬ 
bility  available  to  the  user  demands  that  the  Corporation  recognize 
interface  limitations  and  work  to  establish  how  the  problems  will  be 
overcome  and  by  whom.  MITRE  people  cannot  content  themselves 
with  being  experts  on  the  particular  subsystem  for  which  they  have 
system  engineering  responsibility.  They  must  also  be  experts  on  the 
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other  subsystems  that  contribute  to  the  overall  mission  capability. 
MITRE  has  had  system  engineering  responsibility  tor  many  of  these 
systems,  and  with  that  responsibility  comes  a  bigger  one — systems 
engineering — to  see  to  it  that  all  the  pieces  fit  together  and  that  the 
overall  capability  is  the  best  that  can  be  achieved  on  a  reasonable 
schedule  and  at  reasonable  cost.  Attention  to  the  interoperability  of 
the  various  systems  that  constitute  an  overall  mission  system  capabil¬ 
ity  is  an  important  part  of  MITRE's  systems  engineering  work. 

Many  groups  other  than  the  user  command  and  the  system  pro¬ 
gram  development  agency  are  involved  in  these  programs.  In  each  of 
these  agencies,  many  different  people  have  a  voice  on  the  program, 
and  there  is  considerable  turnover  in  these  personnel  over  the  life  of  a 
typical  acquisition  program.  Congress  approves  funding,  and  congres¬ 
sional  staff  are  especially  active  in  trying  to  understand  each  pro¬ 
gram.  They  advise  the  members  of  Congress  on  plans  for  new 
programs  and  on  the  progress  or  lack  of  it  on  existing  programs. 
Congressional  uction  may  have  profound  impact  on  a  program — it 
might  be  canceled  altogether,  for  e.xample. 

Congressional  actions  may  have  very  substantial  impact  without 
being  lethal,  however.  For  example,  when  funds  are  cut  and  produc¬ 
tion  rates  are  substantially  reduced,  as  happened  with  AW  ACS,  the 
unit  cost  goes  up  dramatically,  and  the  planned  total  buy  may  have 
to  be  reduced  with  corresponding  impact  on  the  capability  available 
to  the  user.  Later  in  the  program,  those  who  are  uninformed  about 
the  background  want  the  development  agencies  to  explain  why  unit 
costs  went  up,  or  why  they  did  not  deliver  the  total  fleet.  This  is  one 
of  the  hazards  of  the  business.  As  systems  engineer,  MITRE  must  be 
continually  alert  to  the  actions  of  Congress  and  the  many  other  DOD 
and  service  groups  as  they  might  affect  the  program.  Further,  the 
Corporation  must  advise  the  program  director  on  actions  required  to 
avoid  undue  impact  on  the  program  by  other  agencies.  When  changes 
are  directed,  MITRE  must  help  the  program  director  make  them 
while  minimizing  the  impact  on  the  capability  achieved,  schedule,  and 
cost.  To  accomplish  this  responsibility,  it  is  important  that  iMlTRE 


interact  with  the  key  people  in  government  and  industry.  Insights 
gained  in  this  way  can  be  very  useful  in  helping  to  formulate  advice 
that  the  Corporation  gives  to  a  program  director  on  how  to  handle  a 
particular  problem  or  opportunity  that  may  arise  on  the  system 
acquisition  program. 

Many  of  the  changes  that  the  environment  forces  on  a  program  are 
less  obvious  than  those  that  result  from  congressional  funding  cuts. 
However,  their  impact  on  performance,  schedule  and  cost  can  be  just  as 
serious.  The  government  may  impose  new  message  standards  or  new 
communications  protocols  on  the  program.  Each  action  may  be  a  good 
thing  to  do,  but  might  have  tremendous  impact  on  an  existing  program. 

When  changes  are  necessary,  for  whatever  reason,  MITRE  must 
work  with  the  other  program  participants  to  identify  the  modifica¬ 
tions  in  the  program  that  must  be  made.  One  very  important  consid¬ 
eration  is  the  existing  industrial  contracts.  Which  ones  must  be 
changed  and  how?  This  area  can  be  a  major  impediment  to  rapid 
reaction  to  required  change.  Among  the  other  questions  to  be  an¬ 
swered  by  the  participants  are:  What  is  the  impact  on  system  perfor¬ 
mance?  What  are  the  schedule  and  cost  implications?  Too  often,  in  a 
rush  to  be  responsive,  one  tries  to  accommodate  directed  changes 
within  the  existing  program.  MITRE  must  recognize  when  such  an 
approach  will  result  in  less  system  performance  or  more  time  and 
money.  Directed  changes  that  have  significant  impact  on  the  program 
need  to  be  so  identified  and  their  performance,  cost,  and  schedule 
implications  clearly  described  at  the  time  the  changes  are  directed. 
The  interest  here  in  early  identification  of  the  impact  of  change  is  not 
to  say,  “I  told  you  so”  later.  Rather,  it  is  in  the  interest  of  identifying 
what  should  be  done  to  make  the  best  of  the  situation  early  enough 
to  avoid  undesirable  compromises  in  capability,  schedule,  or  cost.  It 
is  simply  the  professional  way  for  MITRE  to  conduct  itself  in  its 
dedication  to  achieving  the  capability.  When  necessary,  the  Corpora¬ 
tion  should  work  with  the  program  director  to  have  his  or  her 
direction  and  funding  modified  to  be  consistent  with  the  change 
being  imposed. 
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In  accommodating  change,  one  should  also  consider  the  possibility  of 
packaging  a  number  of  individual  changes  together.  Preplanned  "bkKK” 
changes  may  have  advantages  in  both  the  operational  and  acquisition 
areas.  A  system  that  is  in  a  constant  state  of  change  tends  to  lose  opera¬ 
tional  effectiveness.  Similarly,  changes  that  are  made  individually  are 
more  costly  and  take  more  time  than  those  implemented  as  a  package. 
Program  stability  may  be  more  important  than  responsiveness.  MITRF. 
should  keep  this  possibility  in  mind  in  pn  .-iding  advice  to  the  system 
program  director  on  dealing  with  potential  changes. 

Obviously,  agreements  among  all  the  parties  are  not  achieved  once 
and  then  honored  for  the  life  of  the  acquisition  program.  Many  of  the 
persons  involved  change  over  time  and  the  new  people  do  not  necessarily 
think  the  same  way  as  their  predecessors.  As  sy.stems  engineer,  MITRE 
must  continually  be  tuned  to  manges  in  key  personnel  in  other  agencies 
and  to  the  potential  impact  of  those  changes.  Realistically,  there  are 
many  other  factors  that  may  require  changes  in  the  program.  As  noted 
above,  funding  may  change.  The  Corporation  must  be  prepared  to 
recommend  what  action  should  he  taken  as  a  result,  with  all  things 
considered.  But  other  important  things  change,  too.  Perhaps  the  threat 
has  changed,  or  perhaps  the  available  technology  has  improved  to  such 
an  extent  that  the  program  should  modify  its  technical  approach,  or 
maybe  what  was  being  attempted  was  too  ambitious  and  needs  to  be  cut 
back.  .MITRE  has  access  to  government  threat  information  not  generally 
available  to  contractors.  With  that  access  comes  the  responsibility  to  be 
sure  the  nature  of  the  threat  is  continually  assessed  and  the  impact  of  it 
reflected  in  the  program  in  a  timely  way. 

Similarly,  MITRE  is  a  key  technical  player  on  the  government  side 
with  regard  to  availa'''e  technology  and  its  application  to  the  pro¬ 
gram.  The  Corporation  evaluates  the  technical  progress  being  made 
on  the  program,  continually  assesses  new  technology  that  may  be 
coming  available,  and  estimates  its  potential  use  on  the  program.  The 
concern  for  technical  progress  and  the  knowledge  of  potentially 
applicable  technology  are  two  major  thrusts  of  MITRE’s  systems 
engineering  work.  These  two  areas  must  be  monitored  closely  by 


MITRE  throughout  the  life  of  a  program.  Further,  the  Corporation 
must  take  the  initiative  with  the  program  director  to  be  sure  that 
their  impact  on  the  program  is  both  understood  and  accommodated 
when  appropriate.  However,  one  must  resist  the  allure  of  new  ind 
potentially  risky  technology  when  the  available  technology  is  ad¬ 
equate  for  the  purpose. 

Another  aspect  of  the  environment  in  which  an  acquisition  takes 
place  is  ihe  competition  that  exists  among  development  programs. 
Even  a  program  that  is  providing  the  necessary  capability  on  schedule 
and  at  planned  costs  is  not  exempt  from  this  competition.  As  every¬ 
one  appreciates,  defense  programs  are  in  competition  for  funds  with 
other  government  needs,  such  as  social  services.  Within  the  money 
available  for  defense,  there  are  many  conflicting  demands.  Again, 
agreements  made  last  year  may  no  longer  hold  this  year.  This  is  the 
nature  of  the  business  and  as  systems  engineer,  MITRE  must  help  the 
program  director  anticipate,  avoid,  or  react  to  changes  that  affect  the 
capability  to  be  delivered. 

Clearly,  the  earlier  in  a  program  that  the  need  for  change  is  recog¬ 
nized,  the  better.  The  longer  a  program  proceeds  down  a  certain 
path,  the  more  entrenched  people  will  get  in  what  they  are  doing  and 
the  more  difficult  it  will  be  to  persuade  them  to  change.  Also,  the 
more  resources  are  expended  in  a  certain  direction,  the  more  that 
may  be  wasted  if  that  direction  is  changed.  MITRE  should  not  only 
understand  what  is  happening  at  any  given  time  on  a  program,  but 
also  extrapolate  ahead.  Anticipating  the  need  for  change,  and  the 
earlier  the  better,  is  a  key  MITRE  role. 

Suffice  it  to  say  that  the  environment  within  which  a  system  acqui¬ 
sition  program  '•akes  place  is  very  complicated.  Understanding  that 
environment  in  a  firsthand  way  and  reflecting  it  in  the  advice  given  to 
the  system  program  director  are  major  MITRE  systems  engineering 
responsibilities.  Being  expert  on  the  details  of  the  program  in  ques¬ 
tion  is  not  enough.  To  be  effective,  the  technical  staff  at  MITRE  must 
put  its  immediate  work  in  the  larger,  real-world  context. 


ChapterFour 
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mitre's  Qualifications  and  Approach 


The  MITRE  Corporation  is  uniquely  qualified  to  perform  as  sys¬ 
tems  engineer  for  some  of  the  government’s  most  challenging  C^I 
acquisition  programs.  As  discussed  in  this  chapter,  MITRE’s  unique¬ 
ness  is  derived  in  part  from  its  corporate  form,  a  highly  qualified 
technical  staff,  and  a  deep  appreciation  of  the  operational  capabilities 
desired  by  the  government.  The  Corporation’s  35  years  of  experience 
in  the  systems  engineering  role  and  the  dedication  of  the  management 
and  staff  to  achieving  the  required  systems  capabilities  are  other 
factors  in  that  uniqueness.  The  chapter  concludes  with  a  review  of 
how  mitre's  background  and  experience  are  applied  to  some  of  the 
more  sensitive  systems  engineering  issues  that  arise  on  almost  every 
C’l  systems  acquisition  program. 


Corporote  Form 

MITRE  and  the  Air  Force  have  a  special  relationship.  The  Corpo¬ 
ration  was  formed  in  1958  by  the  Massachusetts  Institute  of  Tech¬ 
nology  (MIT)  at  the  request  of  the  Air  Force,  after  it  was  determined 
that  the  people  at  MIT’s  Lincoln  Laboratory  were  uniquely  qualified 
to  assume  the  system  engineering  role  for  the  nationwide  implementa¬ 
tion  of  the  SAGE  continental  air  defense  system.  Lincoln  Laboratory 


was  responsible  for  the  initial  d^-ifen  of  the  system,  and  as  the  system 
moved  into  a  production  and  implementation  phase,  MIT  recom¬ 
mended  that  subsequent  system  engineering  responsibility  be  trans¬ 
ferred  to  a  non-academic  institution. 

Building  the  SAGE  system  involved  almost  every  major  defense 
contractor  in  the  United  States.  In  looking  for  an  alternative  to  MIT 
for  SAGE  system  engineering,  the  Air  Force  recognized  that  the  sys¬ 
tem  engineering  organization  could  not  carry  out  that  role  without 
potential  conflict  of  interest  if  the  organization  was  also  eligible  for 
contracts  to  build  parts  of  the  system.  Since  the  industrial  defense 
contractors  had  profit  as  a  meaningful  motive,  and  since  the  highest 
chance  for  profit  was  in  manufacturing,  not  system  engineering, 
major  industrial  firms  declined  to  accept  the  system  engineering  role 
under  the  Air  Force  requirement  that  they  could  not  both  assume  that 
role  and  continue  to  build  portions  of  the  system. 

After  considering  a  variety  of  industrial,  governmental,  and 
other  alternatives,  the  Air  Force  concluded  that  only  the  people  at 
Lincoln  Laboratory  were  qualified  to  carry  on  the  SAGE  work  and 
asked  the  university  to  help  them  retain  that  talent.  In  response  to 
that  request,  MITRE  was  formed  under  the  sponsorship  of  the  Air 
Force.  Significant  numbers  of  the  Lincoln  Laboratory  personnel 
who  had  been  working  on  SAGE  transferred  to  the  new  corpora¬ 
tion  in  January  1959.  In  part,  then,  it  was  the  uniqueness  of  the 
knowledge  and  skill  of  the  people  involved  that  led  to  the  forma¬ 
tion  of  MITRE. 

MITRE’s  unique,  long-term  relationship  with  the  Air  Force  on 
systems  engineering  for  C’l  systems  was  reaffirmed  in  1985  in  a 
sponsoring  agreement  signed  by  the  Commander  of  the  Air  Force 
Systems  Command  and  the  Chairman  of  the  MITRE  Board  of 
Trustees."*-  Over  time,  the  Corporation’s  role — with  the  Air  Force’s 
support — has  been  extended  to  similarly  serve  other  parts  of  the 
government,  as  well. 

■*-  Biisic  Principles  Governing  Air  force  MITRE  Relationships,  October  23,  1985. 


The  rules  under  which  the  Corporation  operates  were  formulated  in 
the  very  beginning  to  permit  performing  in  the  unique  systems  engineer¬ 
ing  role.  MITRE  configured  itself  to  minimize  the  chances  for,  or  appear¬ 
ances  of,  conflict  of  interest  either  for  the  Corporation  as  a  whole,  or  for 
individual  staff.  All  parts  of  the  Corporation's  operation  are  open  to 
government  review.  MITRE  personnel  are  appropriately  constrained 
with  regard  to  personal  outside  activities.  As  a  nonprofit  corporation, 
MITRE  does  not  pay  taxes  on  its  income.  But  MITRE  is  also  a  not-for- 
profit  corporation,  a  term  used  to  make  clear  that  by  charter  and  con¬ 
tract  MITRE  does  no  manufacturing,  will  not  accept  work  from 
commercial  or  industrial  organizations,  and  will  not  formally  enter  bid 
competitions  with  profit-making  corporations.  These  factors  permit 
xMITRE  to  operate  in  as  conflict-free  a  way  as  possible.  They  are  not 
limitations;  they  help  the  Corporation  to  be  effective  in  the  systems 
engineering  role  between  government  and  industry. 

Over  the  last  45  years,  a  set  of  formally  recognized  organizations 
devoted  to  public  affairs  and  run  either  privately  or  by  universities, 
has  come  into  being.  They  now  represent  about  five  percent  of  the 
government’s  total  research  and  development  (R&D)  budget.  In 
1968,  the  President’s  Office  of  Science  and  Technology  officially 
designated  such  organizations  as  Federally  Funded  Research  and 
Development  Centers  (FFRDCs).  The  DOD  had  previously  chosen  to 
refer  to  the  FFRDCs  for  which  it  was  responsible  as  Federal  Contract 
Research  Centers,  or  FCRCs.  In  1984,  the  Office  of  Management  and 
Budget  (OMB)  in  the  Executive  Office  of  the  President  of  the  United 
States  issued  a  policy  letter  formalizing  FFRDCs  as  a  new  source  for 
federal  R&D,  adding  to  the  three  existing  sources:  industry, 
academia,  and  the  government  itself. 

The  part  of  MITRE  that  works  on  DOD  CM  systems  is  an  FFRDC. 
The  implications  of  being  an  FFRDC  are  amply  and  well  discussed  in 
a  MITRE  paper  by  Dr.  Norman  Waks.^’  In  fact,  MITRE  has  always 
operated  in  such  a  way  that  it  fulfilled  OMB’s  criteria  for  an  FFRDC, 


■’^N.  Wales,  The  Uniqueness  of  the  M/TR£  Corporation^  M87-1.S,  The  MITRF 
C.'orporation,  Bedford,  XtA,  \farch  1987. 


even  before  the  category  was  established.  From  the  beginning, 

MITRE  was  specifically  constituted  in  ways  that  made  it  possible  to 
fulf’’’  c  Corporation’s  conflict-free  systems  engineering  role.  The 
original  MITRE  charter  called  for  the  Corporation  to  operate  “in  the 
public  interest.”  The  Corporation  had  been  performing  R&D  work 
for  the  government  since  1959.  Therefore,  when  such  categories  were 
established,  MITRE  became  both  an  EERDC  and  an  ECRC.  To  fur¬ 
ther  narrow  the  class  of  corporations  to  which  the  Corporation 
belongs,  only  two  of  the  FCRCs  perform  in  the  systems  engineering 
role:  MITRE  for  CM  systems  for  DOD,  and  Aerospace  Corporation 
for  systems  developed  by  the  Space  Systems  Center  of  the  AEMC. 

The  other  FERDCs  fall  into  two  other  categories:  R&D  laboratories 
or  Studies  and  Analyses  Centers.  MITRE’s  corporate  form  and  work 
area  are  unique  among  industry,  government,  academia,  and  even  the 
other  FFRDCs/ECRCs. 

The  government  contracts  with  MITRE  through  directed  awards, 
based  on  the  Corporation’s  unique  qualifications  to  carry  out  the 
required  work.  Avoiding  the  need  to  compete  is  not  the  purpose  of 
directed  awards;  directed  awards  are  used  because  the  government 
will  not  permit  MITRE  to  enter  formal  competitions.  By  refusing  to 
permit  MITRE  to  bid  against  profit-making  industry,  the  government 
protects  and  reinforces  MITRE’s  conflict-free  status.  The  awards 
directed  to  MITRE  may  be  based  on  a  sole-source  determination  in 
which  the  government  establishes  the  best  possible  source  for  the 
professional  service  and  then  directs  the  award  to  that  source.  Al¬ 
though  MITRE  does  not  formally  compete,  clearly  the  Corporation 
must  be  competitive — in  technical  capability  and  cost  of  doing  busi¬ 
ness — to  be  uniquely  qualified. 

As  an  independent  contractor,  MITRE  is  neither  part  of  govern¬ 
ment  nor  part  of  industry.  As  noted  by  Dr.  Waks,  in  1972,  the  con- 
gressionally-initiated  Commission  on  Government  Procurement 
recognized  the  “independent  perspective”  of  the  FFRDCs  and  cited 
the  government’s  need  for  FFRDCs  to  “look  over  its  [the  govern¬ 
ment’s]  shoulder”  in  areas  of  great  consequence.  The  government 
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wants  MITRE’s  best  advice,  and  the  Corporation  is  dedicated  to 
providing  it. 

As  noted  in  Volume  1  of  Quality  Assurance  at  MITRE, 

Tasks  to  be  accomplished  by  MITRE  are  assigned  only  when  the 
role  is  appropriate.  MITRE  is  not  used  if  an  appropriate  govern¬ 
ment  capability  exists  or  if  industry  can  do  the  job  as  effectively 
and  without  conflict  of  interest.'''* 

Specific  criteria  applied  by  the  government  in  deciding  whether  to 
assign  work  to  MITRE  are  described  below.  The  above  reference  goes 
on  to  note  that  this  sort  of  scrutiny  helps  ensure  a  technically  chal¬ 
lenging  and  important  work  program  for  MITRE.  Rather  than  view¬ 
ing  this  process  as  a  limitation  on  what  MITRE  may  do,  one  should 
appreciate  that  it  forms  the  foundation  for  the  strengths  of  the  Cor¬ 
poration.  It  is  also  the  foundation  for  the  special  relationship  that 
exists  between  MITRE  and  its  sponsors,  including  the  Air  Force. 

The  character  of  the  MITRE  work  covered  by  the  contract  with 
ESC  is  described  in  ESC  Regulation  80-1  (ESCR  80-1).  This  docu¬ 
ment  describes  MITRE  as  an  independent  contractor,  responsible  for 
the  management  of  its  own  affairs,  and  notes  that  this  independence 
is  necessary  for  the  Corporation  to  maintain  the  objectivity  valued  by 
the  government.  System  engineering  is  called  the  preferred  MITRE 
role,  and  in  this  role  the  regulation  states  that  MITRE  will; 

(a)  take  broad  initiatives  on  all  technical  matters  pertaining  to 
the  program; 

(b>  provide  positions  on  all  technical  matters  of  interest  to  the 
Government  Program  Manager; 

(c)  adopt  a  broad  view  transcending  the  strictly  technical  aspects 
that  contribute  to  a  program  decision;  and 

(d)  keep  the  Program  Office  advised  of  its  activities.'*'' 

ESCR  80-1  then  goes  on  to  discuss  typical  systems  engineering  activi¬ 
ties.  It  also  notes  the  sort  of  tasks  that  should  not  be  assigned  to  MITRE; 

Quality  Assurance  at  MITRE,  Vol.  1,  .\188-39,  The  MITRE  Corporation, 
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■*'  Utilization  of  MITRE  Support,  ESC  Regulation  80-1,  December  1991. 


for  example,  MITRE  is  not  to  be  used  for  routine  technical,  administra¬ 
tive,  or  management  tasks,  and  use  of  MITRE  staff  to  augment  the  Air 
Force  technical  staff  is  specifically  precluded.  More  precisely, 

MITRE  is  assigned  responsibility  for  specific  jobs  it  has  contrac¬ 
tually  accepted  and  will  not  provide  personnel  for  assignment 
and  direction  by  the  government.'*^ 

ESCR  80-1  also  reiterates  and  expands  on  the  criteria  to  be  used  in 
determining  the  appropriateness  of  a  piece  of  work  for  assignment  to 
MITRE  to  include: 

(1)  freedom  from  bias  due  to  preference  in  design,  hardware, 
or  approach; 

(2)  need  for  industry  proprietary  information; 

(i)  access  to  industry  proposals; 

(4)  need  for  extensive  background  information; 

(5)  need  for  state-of-the-art  information  from  government 
laboratories; 

(6)  access  to  Air  Force  planning  information; 

(7)  access  to  intelligence; 

(8)  need  for  outstanding  specialists  in  specific  fields; 

(9)  need  for  diversified  skills; 

(10)  performance  of  technology  base  functions; 

(11)  continuity  of  effort; 

(12)  need  for  special  facilities  and 

(13)  need  for  fast  response 

Each  of  these  areas  is  described  further  in  the  regulation.  These  crite¬ 
ria  serve  to  ensure  that  MITRE  is  properly  used  by  the  government  and, 
since  the  MITRE  resource  is  limited,  help  to  ensure  that  the  Corporation 
is  employed  on  the  most  demanding  government  program^. 

As  mentioned  in  Chapter  3,  MITRE’s  work  on  important  govern¬ 
ment  programs  is  organized  into  projects;  for  each  project,  there  is  a 
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project  leacier  who  supports  a  corresponding  person  in  the  govern¬ 
ment,  usually  called  the  program  director  or  project  officer.  The 
project  leader  is  responsible  for  the  day-to-day  activities  of  MITRE 
personnel  assigned  to  the  project.  For  a  typical  project,  the  MITRE 
responsibilities  are  delineated  in  the  TO&P.  Each  TO&P  is  incorpo¬ 
rated  by  reference  in  the  formal  contract  between  MITRE  and  the 
government  agency  involved.  The  contract,  which  may  cover  a  large 
number  of  individual  projects,  is  between  the  Corporation  and  the 
government  agency,  not  between  the  project  leader  (as  a  representa¬ 
tive  of  the  Corporation)  and  the  government  program  director. 

MITRE  does  not  contract  for  the  personal  services  of  its  staff; 
rather,  the  Corporation  contracts  for  the  products  described  in  each 
TO&P  and  agrees  to  provide  those  products  under  the  terms  of  the 
relevant  contract.  The  project  leader  and  the  other  personnel  assigned 
to  work  on  the  project  are  responsible  to  corporate  management  for 
accomplishing  the  work  described  in  the  TO&P.  The  Corporation  is 
responsible  for  delivering  the  required  products  under  the  terms  of 
the  contract. 

In  addition  to  selecting  a  capable  project  leader,  management 
above  the  project  leader  has  a  responsibility  to  ensure  that  appropri¬ 
ate  personnel  are  assigned  and  that  adequate  resources  are  available 
to  carry  out  the  work.  Management  must  also  agree  that  the  work  to 
be  performed  is  appropriate  for  MITRE.  Management  has  the  normal 
responsibility  for  in-process  review  of  the  work  to  ensure  it  is  respon¬ 
sive  to  the  customer’s  needs  and  satisfactorily  performed.  MlTRE’s 
management  is  an  active  participant  in  the  work  done  on  each 
project.  Project  planning  must  provide  time  for  their  reviews  and  for 
actions  that  stem  from  those  reviews. 

MITRE  is  willing  and  very  quickly  able  to  modify  its  work  pro¬ 
gram  when  requested  to  do  so  by  the  Air  Force.  The  Corporation  has 
experience  in  most  areas  of  CM  of  interest  to  the  United  States.  When 
requested,  with  an  absolute  minimum  of  negotiation  and  paperwork, 
MITRE  can  quickly  reduce  work  in  one  area  to  assume  higher  prior¬ 
ity  work  in  another  one. 


Since  there  are  government-imposed  controls  on  the  rate  of  growth 
of  MITRE,  and  natural  controls  on  such  growth  in  any  case,  one 
finds  that  the  specifics  of  the  MITRE  work  program  are  constantly 
changing  in  response  to  ESC  needs.  The  rate  and  the  extent  of  these 
changes  are  far  beyond  what  could  be  handled  through  normal, 
competitive  industrial  means.  This  would  be  so  even  if  industry  were 
willing  and  able  to  meet  all  the  other  constraints  placed  on  MITRE. 
The  required  changes  are  facilitated  by  the  management  processes 
used  by  ESC  and  MITRE.  They  are  possible  because  of  the  breadth 
and  experience  of  the  Corporation’s  staff. 

MITRE’s  designation  as  an  FFRDC  does  not  relieve  the  Corpora¬ 
tion  from  many  of  the  challenges  that  face  any  business  or  industry. 
The  Corporation  has  no  guarantee  of  work  from  any  government 
organization.  As  an  FFRDC,  MITRE  will  not  enter  bid  competitions 
with  industry.  The  Corporation’s  viability  as  a  systems  engineering 
organization  is  based  on  the  quality  and  cost  of  the  work  it  performs. 
The  Corporation  is  subject  to  all  the  market  forces  that  govern  the 
establishment  of  a  quality  work  force  at  competitive  costs.  To  attract 
a  quality  staff,  the  work  must  be  challenging  and  rewarding,  the 
facilities  matched  to  accomplishing  the  work,  and  the  salaries  and 
benefits  competitive  with  industry.  To  be  competitive  with  other 
sources  of  systems  engineering,  the  work  must  be  well  done  and  the 
costs  as  low  or  lower  than  potential  alternatives  for  equivalent  work. 
MITRE  must  be  vigilant  about  the  quality  of  the  work  performed  and 
about  the  cost  of  its  own  operation. 

In  most  cases,  MITRE  is  funded  by  an  agency  that  is  responsible 
for  development  of  a  system  or  capability.  A  program  director  (or  an 
analogous  person)  is  the  agency  representative  responsible  for  the 
day-to-day  conduct  of  the  development  program,  including  the 
MITRE  work  on  that  program.  Clearly,  MITRE  must  respond  to  the 
needs  of  the  program  director.  The  development  agency  is  responsible 
for  delivering  the  system  to  a  user  who  will  combine  the  system  with 
other  existing  ones,  and  perhaps  new  systems,  to  achieve  a  required 
operational  capability.  In  some  sense,  then,  the  user  is  the  ultimate 
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customer  of  both  the  project  officer  and  MITRE.  At  the  same  time, 
there  is  an  organizational  hierarchy  within  the  government  that  has 
individual  development  agencies  reporting  through  a  series  of  levels 
to  a  service  headquarters,  for  example.  The  project  officer  is  respon¬ 
sible  for  satisfying  the  demands  of  these  groups,  as  well  as  for  meet¬ 
ing  the  needs  of  the  user.  MITRE  is  responsible  for  helping  the 
project  officer  to  do  that. 

MITRE  knows  it  has  clear  obligations  to  the  program  director, 
that  the  program  director  provides  the  funds  for  systems  engineering, 
and  that  the  program  director  has  the  ultimate  responsibility  for 
meeting  the  user  requirements.  The  MITRE  contract  covering  the 
work  performed  for  the  Air  Force  is  issued  by  ESC.  Each  acquisition 
program  director  decides  what  work  he  or  she  would  like  MITRE  to 
do,  negotiates  the  terms  of  that  work  with  MITRE,  and  provides  the 
necessary  funding  to  ESC  for  application  to  the  MITRE  contract.  As 
a  result,  the  Corporation  has  an  obligation  to  ESC,  to  AFMC,  and  to 
each  of  the  individual  program  directors  for  the  performance  of  the 
work  defined  in  the  contract.  MITRE  believes  that  meeting  these 
responsibilities  requires  direct  interaction  between  MITRE  staff  and 
management  personnel  and  their  counterparts  in  these  agencies  and  in 
industry. 

MITRE’s  articles  of  incorporation  state  in  part  that  the  Corporation 
was  formed  to  perform  scientific  and  engineering  services  to  enhance 
the  security  of  the  United  States  and  to  otherwise  further  the  public 
interest.  MITRE’s  Board  of  Trustees  is  responsible  for  ensuring  that  the 
Corporation  acts  accordingly.  The  Board  actively  reviews  the  work 
that  MITRE  proposes  to  do  before  that  work  is  accepted,  to  be  sure  it 
is  in  the  public  interest  and  appropriate  for  an  FFRDC.  For  the  same 
reasons,  the  Board  also  does  in-depth  reviews  of  the  manner  in  which 
the  contracted  work  is  performed,  with  particular  interest  in  the  qual¬ 
ity  of  that  work.  These  reviews  affect  what  MITRE  does  and  how. 

MITRE’s  management  is  committed  to  ensuring  that  the  Corpora¬ 
tion  effectively  meets  both  the  explicit  requirements  of  the  TO&P  in 
support  of  each  program  director,  and  the  implicit  commitments  to 


higher  headquarters  and  to  the  user.  Management  is  also  responsible 
for  ensuring  that  appropriate  resources  are  assigned  to  each  program 
and  that  those  resources  effectively  deliver  the  products  described  in 
the  TO&Ps.  To  do  that,  management  must  be  well  informed  on  each 
program  and  must  use  its  experience  and  knowledge  to  direct  the 
work  of  the  staff. 

As  described  in  Chapter  3,  each  program  is  conducted  in  a  very 
expansive  and  volatile  environment.  It  is  therefore  important  for 
management  to  ensure  that  the  staff  is  well  informed  on  all  factors 
that  may  have  an  impact  on  the  success  of  an  acquisition  program.  In 
turn,  this  dictates  that  management  interacts  with  others  in  industry 
and  government  who  may  have  insights  useful  to  the  program.  Given 
that  such  interactions  are  important  to  ensuring  the  efficacy  of  the 
MITRE  work,  it  is  clearly  incumbent  on  management  to  ensure  that 
such  interactions  take  place  in  ways  that  do  not  undermine — or  even 
appear  to  undermine — the  Corporation’s  contractual  obligations  to 
the  program  directors  and  to  ESC.  This  is  a  delicate  task  for  manage¬ 
ment  that  requires  the  understanding  and  good  faith  of  all  parties. 
This  interaction  between  MITRE  management  and  others  in  govern¬ 
ment  and  industry  can  be  a  very  effective  tool  available  to  help  pro¬ 
gram  directors  conduct  their  programs  successfully. 

MITRE  believes  it  has  obligations  to  ESC,  and  to  higher  headquar¬ 
ters,  that  transcend  the  specific  requirements  of  the  individual 
TO&Ps  incorporated  in  the  contract  with  ESC.  The  Corporation  is 
dedicated  to  meeting  these  obligations  without  compromising  its 
systems  engineering  work  on  individual  acquisition  programs. 

Miriiitoinfflg  0  SkiUed  Tedinkol  Staff 

Both  broad  system  knowledge  and  in-depth  technical  expertise  in 
many  areas  are  required  for  effective  systems  engineering.  One  does 
not  necessarily  expect  to  find  both  in  the  same  person  and  sometimes 
not  even  in  the  same  organization.  However,  MITRE  has  both  skills. 
In  many  organizations,  there  are  groups  that  are  clearly  franchised  as 
technical  and  somehow  distinguished  from  project  personnel.  At 


The  Corporation  is  not  merely 
providing  the  services  of  a  set 
of  people  with  technicol  skills; 
it  provides  corporote  systems 
engineering  support  dedicated 
to  achieving  the  required 
capability  and  provides  that 
support  through  skilled 
technical  people. 


MITRE,  some  staff  is  assigned  to  the  technical  centers,  but  the  major¬ 
ity  of  the  staff  is  not.  Also,  at  any  given  time,  most  of  the  staff  of  the 
technical  centers  is  assigned  to  work  on  acquisition  projects  for  which 
MITRE  has  the  systems  engineering  role.  Whatever  organizational 
approach  one  uses,  there  is  conflict  between  those  who  tend  to  con¬ 
sider  the  technical  first  and  everything  else  second,  and  those  who 
feel  that  the  demands  of  the  project  may  require  some  compromise  in 
the  technical  area.  This  conflict  sometimes  results  in  project  personnel 
being  referred  to  as  generalists.  The  implication  is  that  the  generalist 
has  less  technical  skill.  There  is,  of  course,  no  a  priori  reason  for  that 
to  be  so.  An  interesting  quotation  on  this  subject  is  found  in  R.E. 
Machol’s  book: 

The  system  engineer  must  he  a  generalist  as  distinguished  from 
a  specialist,  hut  he  must  not  he  a  dilettante.  The  ideal  system 
engineer  is  .  .  .  “T-shaped".  .  .  hroad,  hut  deep  in  one  field;  the 
depth  is  provided  hy  scholarly  experience.'*’^ 

MITRE  as  a  corporation  is  dedicated  to  maintaining  the  highest 
level  of  technical  competence  possible  to  fulfill  its  systems  engineering 
role  for  the  government.  High  technical  competence  is  required  if 
MITRE  is  to  “take  broad  initiatives  on  all  technical  matters  pertain¬ 
ing  to  the  program”  and  to  “provide  positions  on  all  technical  mat¬ 
ters  of  interest”  as  required  by  ESCR  80-1.  That  is  an  awesome  task 
when  one  considers  the  wide  range  of  systems  for  which  the  govern¬ 
ment  has  asked  MITRE  to  assume  systems  engineering  responsibility. 
It  is  also  one  that  the  Corporation  takes  very  seriously.  It  is  a  major 
factor  in  the  Corporation’s  approach  to  systems  engineering.  The 
success  of  mitre’s  systems  engineering  is  measured  by  the  quality  of 
the  system  that  results.  The  Corporation  is  not  merely  providing  the 
services  of  a  set  of  people  with  technical  skills;  it  provides  corporate 
systems  engineering  support  dedicated  to  achieving  the  required 
capability  and  provides  that  support  through  skilled  technical  people. 
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Among  MITRE  project  personnel,  one  finds  all  three  of  the  essen¬ 
tial  ingredients  of  successful  systems  engineering:  in-depth  under¬ 
standing  of  and  appreciation  for  the  needs  that  the  system  is  to 
satisfy;  current  knowledge  of  and  the  skill  to  apply  the  appropriate 
technologies;  and  a  persistence  and  ability  to  apply  these  two  skill 
areas  in  the  real  world  for  the  achievement  of  the  needed  capability. 

The  quality  of  the  MITRE  staff  is  inextricably  intertwined  with  the 
quality  of  the  MITRE  work  program.  High  quality  work  attracts  and 
retains  highly  skilled  people.  Highly  qualified  technical  personnel 
attract  challenging  work.  As  discussed  earlier  in  this  book,  there  are  a 
series  of  checks  made  by  the  government  and  the  Corporation  on 
each  new  piece  of  proposed  work  to  be  sure  that  work  is  appropriate 
for  assignment  to  MITRE.  Those  checks  help  to  ensure  a  high  quality 
work  program.  As  will  be  discussed,  the  Corporation  has  a  number 
of  mechanisms  to  ensure  that  the  quality  of  the  staff  matches  well 
with  the  challenge  of  the  work  program. 

The  corporate-wide  efforts  to  maintain  a  high  quality  technical 
staff  are  discussed  in  some  detail  in  a  MITRE  paper  on  how  the 
Corporation  achieves  quality  assurance  on  the  work  of  the  staff. ■*'* 
These  efforts  have  many  dimensions.  One  concerns  the  hiring  and 
retention  of  skilled  technical  personnel.  The  data  provided  in  Volume 
2  of  the  quality  assurance  paper  indicates  that  MITRE  is  a  balance  of 
experienced  and  newly  trained  technical  personnel.  The  number  of 
staff  is  approximately  equal  for  each  5-year  increment  of  years  of 
experience  from  0-4  years  to  20-24  years.  Those  with  more  than  25 
years  of  experience  number  about  twice  the  size  of  each  of  these  5- 
year  groupings.  Over  70  percent  of  the  staff  has  more  than  10  years 
of  experience.  This  mix  of  experience  levels  provides  a  balance  be¬ 
tween  experience  and  recent  education  and  training  that  is  well  suited 
to  helping  MlTRE’s  customers  exploit  the  latest  technology. 

In  the  past,  MITRE  has  increased  the  size  of  its  professional  staff 
to  satisfy  the  government’s  requests  for  additional  systems  engineering 
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support.  In  that  process,  the  Corporation’s  hiring  standards  have 
remained  high.  Less  than  one  percent  of  the  resumes  submitted  to 
MITRE  result  in  the  person  being  hired.  Over  60  percent  of  those 
hired  come  from  industry  and  provide  the  latest  in  industrial  experi¬ 
ence,  while  approximately  15  percent  are  new  college  graduates 
trained  in  the  newest  available  technologies. 

Well  over  50  percent  of  MITRE  staff  has  advanced  degrees  and  the 
mix  is  representative  of  systems  engineering  tasks,  as  opposed  to 
what  might  be  required  in  a  research  laboratory.  The  degree  fields 
included  span  all  those  necessary  for  MJTRE’s  systems  engineering 
work  including  electrical  engineering,  other  engineering,  computer 
science,  the  physical  sciences,  and  mathematics. 

The  high  demand  for  the  Corporation’s  services  has  been  one 
indicator  of  the  quality  of  its  staff.  However,  there  are  others.  Each 
year,  MITRE  publishes  a  set  of  documents  that  record  the  profes¬ 
sional  activities  of  the  staff,  incli:ding  those  that  are  outside  the  work 
done  directly  on  a  customer  project.'"  These  include  membership  in 
professional  societies,  publications,  participation  in  technical  confer¬ 
ences,  work  on  standards  and  other  committees,  teaching  appoint¬ 
ments,  and  honors  and  awards.  The  accomplishments  reflect  well  on 
the  technical  ability  of  the  MITRE  staff.  This  sort  of  recognition  of 
MITRE  people  m  the  general  professional  community  is  another 
indicator  of  the  high  quality  of  the  Corporation’s  technical  staff. 

In  addition  to  attracting  high  quality  staff,  the  Corporation  also 
does  a  number  of  things  to  retain  that  staff  and  to  keep  it  current  on 
the  relevant  technologies.  The  .MITRE  Institute  provides  for  continu¬ 
ing  education  for  all  MITRE  employees.  Under  this  corporate-spon¬ 
sored  program  administered  by  the  Institute,  all  employees  are 
eligible  for  educational  assistance  through  external  courses  at  accred¬ 
ited  institutions.  The  Institute  also  provides  a  number  of  in-house 
courses  that  are  designed  to  meet  MlTRE’s  unique  needs  and  are 
unavailable  elsewhere.  Other  Institute  activities  include  courses  in 
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proFessional  development,  management  development,  project  budget¬ 
ing,  and  technical  docume.itation. 

Under  the  Institute,  the  Corporation  sponsors  a  Masters  Degree 
Program  in  Science  and  Engineering  and  offers  extensive  on-site 
a\  I  lability  of  televised  courses  from  universities  locai  to  MITRE’s 
major  locations.  The  MITRE  Institute  also  sponsors  a  Distinguished 
Lecturer  Series  in  which  distinguished  individuals  from  the  profes¬ 
sional  and  scientific  communities  make  presentations  at  MITRE  on 
subjects  that  both  stimulate  the  technical  skills  of  the  staff  and 
broaden  the  context  within  which  the  staff  makes  technical  judg¬ 
ments.  Recently,  the  Institute  initiated  another  program,  the  National 
Defense  Series,  in  which  senior  people  from  outside  of  MITRE  pro- 

le  the  staff  with  perspectives  on  national  defense  issues  and  plans 
Miiportant  to  the  MITRE  work. 

A  second  major  corporate  approach  to  maintaining  the  technical 
strength  of  the  staff  is  the  MITRE  Technology  Program.  The  MITRE 
Technology  Program  is  partially  funded  from  fee  under  the  MITRE 
Sponsored  Research  (MSR)  Program  and  partially  from  separate 
sponsor-funded  technology  projects.  Some  technology  work  is  also 
done  as  part  of  the  systems  engineering  projects.  Approximately  10 
percent  of  MITRE’s  volume  is  dedicated  to  technology  work.  Man¬ 
agement  identifies  key  general  technology  areas  to  be  pursued  and 
specific  prop'',. '•Is  are  sought  from  the  staff  in  those  areas,  as  well  as 
others.  Management  chooses  from  the  proposed  projects.  Some  are 
then  funded  by  the  sponsors  and  the  remainder  are  supported  by 
MSR  funding.  A  strong  in-house  technology  program  provides  oppor¬ 
tunities  for  the  staff  to  stay  current  and  to  contribute  to  the  state  of 
the  art.  Challenging  technical  work  also  helps  to  attract  additional 
high-quality  professional  staff. 

Another  corporate  method  for  attracting  and  maintaining  a  high 
quality  technical  staff  is  the  use  of  an  alternative  promotional  path 
for  people  who  would  prefer  not  to  assume  management  responsi¬ 
bility  but  would  like  increased  technical  responsibilities  within  the 
Corporation.  The  technical  promotional  path  has  positions  of 


responsibility  corresponding  to  those  ot  the  management  ladder 
within  MITRE.  It  serves  to  provide  paths  for  advancement  for 
highly  qualified  staff  members  who  have  made  significant  contribu¬ 
tions  to  .MlTRE’s  work,  attracts  and  retains  outstanding  technical 
contributors,  and  improves  the  quality  of  technical  work  performed 
by  MITRE.  Members  of  the  technical  ladder  at  all  levels  are 
expected  to  make  significant  contributions  to  MlTRE’s  systems 
engineering  projects. 

Other  corporate  mechanisms  for  ensuring  the  quality  of  MITRE's 
technical  staff  include  information  exchange  programs  with  govern¬ 
ment,  universities,  and  industry.  Personnel  from  these  groups  visit 
MITRE  to  see  its  work,  and  MITRE  personnel  make  reciprocal  visits 
to  government,  university,  and  industry  locations.  The  Corporation 
also  sponsors  symposia  or  large  conferences  each  year  on  topics  of 
interest  to  its  sponsors  and  to  related  industry.  As  already  noted, 
MITRE  staff  members  belong  to  professional  organizations,  regularly 
publish  papers  in  professional  journals,  and  participate  in  special 
government  studies. 

Finally,  MITRE  has  adopted  a  number  of  organizational  mecha¬ 
nisms  designed  to  improve  the  quality  of  the  technical  work  on  the 
acquisition  programs.  The  Corporation  cannot  afford  to  maintain 
large  numbers  of  skilled  technical  people  in  each  of  the  technical 
areas  represented  by  the  programs  for  which  it  has  systems  engineer¬ 
ing  responsibility.  Optimum  use  of  the  technical  resources  across 
MlTRE’s  projects  requires  special  management  attention.  In  a  modi¬ 
fied  matrix  structure,  staff  who  are  experts  in  the  major  specific 
technical  fields  such  as  sensors,  software,  networks,  etc.,  are  assigned 
to  a  corresponding  technical  center.  The  center  directors  in  turn  use 
these  resources  to  provide  support  to  the  acquisition  projects.  This 
approach  increases  the  flexibility  available  to  management,  since 
critical  skills  can  be  applied  more  easily  where  they  are  needed  at  any 
given  time.  It  improves  the  technical  quality  of  the  acquisition  project 
support,  since  the  management  of  the  technical  centers  is  involved  in 
the  work  and  its  review.  The  approach  helps  to  establish  a  critical 


mass  in  the  technical  areas  and  to  make  it  possible  to  provide  special 
tools  to  the  groups  that  could  not  be  provided  if  the  groups  were 
dispersed.  Finally,  this  approach  makes  it  easier  to  attract  top  techni¬ 
cal  people  to  MITRE. 

Related  to  the  technical  centers,  the  Corporation  also  has  a  num¬ 
ber  of  smaller  technical  groups  for  staff  who  are  expert  in  some  of 
the  technical  areas  not  included  among  the  major  technical  centers. 
These  groups  are  officially  recognized  by  the  Corporation  and  pro¬ 
mote  structured  communication,  networking,  and  mutual  support. 

This  section  has  provided  a  very  brief  summary  of  the  MITRE 
activities  designed  to  provide  a  highly  qualified,  professional  technical 
staff  in  support  of  the  Corporation’s  systems  engineering  responsibili¬ 
ties.  It  is  a  challenge  that  MITRE  management  has  taken  very  seri¬ 
ously,  and  one  they  will  continue  to  ensure  is  successfully  met.  Such  a 
staff  is  one  essential  ingredient  of  the  Corporation’s  systems  engineer¬ 
ing  work.  A  second  important  ingredient  is  the  subject  of  the  next 
section  of  this  book — a  technical  staff  with  deep  understanding  of  the 
operational  capability  that  the  government  is  attempting  to  achieve. 

Developing  Staff  Knowledge  of  Operational  Capabilities 

For  the  Corporation  to  be  successful,  the  MITRE  systems  engineer 
must  have  a  deep  appreciation  for  the  operational  capability  that  the 
government  needs — a  thorough,  personal  understanding — not  just  a 
statement  of  what  someone  else  may  say  it  is.  Before  suggesting  how 
the  staff  is  able  to  achieve  such  an  understanding,  it  may  be  helpful 
to  describe,  in  some  detail,  the  extent  to  which  the  staff  must  under¬ 
stand  the  need  and  how  that  understanding  is  employed  in  the  day- 
to-day  systems  engineering  activities. 

Earlier  in  this  book,  it  is  observed  that  the  government  tends  to 
buy  subsystems — a  radar,  or  a  piece  of  communications  equipment, 
or  a  command  center.  The  development  community  provides  the 
subsystems  to  a  using  command  that  must  combine  them  with  other 
existing  subsystems  and  turn  the  collection  into  an  operational  mis¬ 
sion  capability,  such  as  air  defense  or  finding  and  killing  ground 
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targets.  MITRE  has  had  systems  engineering  responsibility  for  most 
of  the  command  and  control-related  items  that  have  been  produced 
over  the  last  35  years  for  the  Air  Force.  An  indication  of  the  span  of 
the  Corporation's  C’l  system  activities  may  be  found  in  a  review  of 
its  first  30  years."  This  paper  lists  well  over  100  different  CT  sys¬ 
tems  for  which  MITRE  has  had  systems  engineering  responsibility. 

In  the  systems  engineering  role,  the  Corporation  has  worked  at 
operating  field  locations  and  helped  to  make  the  systems  function  to 
achieve  mission  capabilities.  MITRE  has  also  had  staff  assisting  in 
this  process  at  major  user  headquarters  locations.  In  both  the  devel¬ 
opment  and  operational  phases,  the  Corporation  has  assisted  in 
providing  effective  interfaces  among  the  systems  of  the  various  mili¬ 
tary  services  and  with  systems  of  other  friendly  nations.  As  a  result  of 
this  background,  within  the  development  community,  MITRE  has  a 
unique  appreciation  for  the  capabilities  that  the  government  is  trying 
to  achieve.  This  observation  is  not  a  negative  reflection  on  any  other 
organization;  it  is  simply  a  statement  that  MITRE  has  had  opportuni¬ 
ties  to  understand  the  essence  of  system  capabilities,  over  longer 
periods  of  time,  and  from  more  perspectives  than  any  other  organiza¬ 
tion,  industrial  or  military.  Neither  is  the  statement  meant  to  confuse 
who  is  responsible  for  achieving  military  capabilities — the  operational 
using  commands.  It  is  also  not  meant  to  cloud  the  clear  responsibili¬ 
ties  of  the  military  service  development  agencies.  The  statement 
merely  means  that  one  of  the  key  foundations  of  MITRE  systems 
engineering  work  is  a  profound  appreciation  for  the  capabilities  to  be 
achieved  and  a  deep  dedication  to  doing  what  is  necessary  to  achieve 
those  capabilities. 

Some  further  discussion  here  is  necessary  to  provide  insight  into 
the  levels  at  which  MITRE  understands  the  capabilities  that  the 
government  is  attempting  to  achieve.  Air  defense  will  be  used  as  an 
example.  Everyone  in  the  business  has  an  idea  about  what  is  meant 
when  someone  says  they  want  an  air  defense  capability — a  capability 
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to  know  one  is  coming  under  air  attack  and  a  means  to  destroy  the 
attackers  before  they  can  damage  the  defended  area.  At  the  next 
level,  one  begins  to  assess  the  characteristics  of  the  threat  and  those 
of  the  existing  friendly  forces  available  to  deter  that  threat.  How 
many  threat  aircraft,  and  what  are  their  range,  altitude,  speed,  and 
weapon-carrying  capabilities?  Do  they  have  standoff  capabilities? 
When  one  begins  to  understand  how  difficult  the  problem  might  be, 
another  round  of  questions  arises  on  what  one  means  by  air  defense 
in  this  case. 

Do  we  merely  mean  to  maintain  sovereignty  of  our  air  space,  with 
the  implied  admission  that  the  system  could  not  handle  any  substan¬ 
tial  air  attack,  or  are  we  intent  on  being  able  to  provide  substantial 
defense  against  a  determined  enemy?  What  is  meant  by  substantial? 
How  big  are  the  attacks  apt  to  be?  How  much  damage  are  we  willing 
to  accept?  Does  the  defense  have  to  be  perfect,  or  only  able  to  handle 
some  major  portion  of  the  threat?  How  much  robustness  should  the 
system  have?  Are  multiple  attacks  expected?  Questions  of  this  sort 
are  many.  The  answers  drive  the  development  programs  in  a  first- 
order  way.  However,  the  answers  are  not  easy  to  come  by.  Many  of 
the  people  involved  do  not  have  the  background  to  answer  them,  and 
important  people  disagree  on  their  views.  Nevertheless,  questions 
such  as  those  above  are  typical  of  many  that  MITRE  considers  in  the 
systems  engineering  role. 

In  addition  to  the  questions  on  broad  operational  requirements, 
there  are  those  related  to  cost,  and  to  the  priority  among  the  many 
competitive  demands  for  government  funding  both  within  DOD  and 
between  DOD  and  other  government  programs.  How  much  are  we 
willing  to  spend  on  air  defense?  Both  acquisition  costs  and  the  own¬ 
ership  costs  are  important.  What  types  and  numbers  of  operational 
and  maintenance  personnel  may  be  made  available  for  the  system? 
What  are  the  logistics  support  implications?  Can  the  air  defense 
assets  contribute  to  other  mission  areas  if  need  be?  Debates  begin  to 
arise  over  whether  one  gets  more  air  defense  for  the  money  by  invest¬ 
ing  in  new  weapons,  sensors,  communications,  or  command  and 


control  centers.  The  politics  of  what  is  built  in  a  particular  congres¬ 
sional  district  may  impinge  on  the  debates.  Other  non-technical 
matters  will  arise  as  issues. 

How  much  capability  do  we  already  have?  What  will  be  provided 
by  other  development  programs  either  in  progress  or  planned  for  the 
future."  Will  other  systems  provide  early  warning  of  attack?  How 
does  one  distribute  the  capability  among  the  subsystems?  Are  the 
subsystems  that  program  direction  says  to  acquire  those  that  will 
provide  the  best  capability  for  the  funds  available?  As  one  proceeds 
down  a  path  such  as  this,  fewer  and  fewer  groups  have  the  back¬ 
ground  or  the  responsibility  for  addressing  such  questions.  Yet  in  its 
systems  engineering  work  on  subsystems  associated  with  air  defense, 
MITRE  considers  such  global  matters,  as  well  as  many  other  more 
detailed  questions. 

As  an  example,  to  provide  air  defense,  one  must  know  when  en¬ 
emy  aircraft  are  approaching  the  defended  area.  The  system  must  be 
able  to  keep  track  of  them  as  they  approach  and  as  they  are  engaged 
by  friendly  forces.  Sensors,  typically  radars,  report  where  the  aircraft 
are  located  and  computers  use  that  data  to  perform  the  air  defense 
function  known  as  automatic  tracking.  However,  the  demands  on  the 
automatic  tracking  function  vary  greatly  as  a  function  of  the  threat 
and  the  weapon  S’  stem  capabilities.  Many  other  questions  arise 
concerning  the  thre.-.':  How  high  or  how  low  and  how  fast  will  the 
enemy  aircraft  fly?  What  maneuver  and  jamming  capabilities  will 
they  have?  Other  types  of  questions  must  be  considered.  At  what 
point  in  the  intercept  and  with  what  accuracy  does  the  tracking 
function  have  to  provide  data  to  the  weapon  platform?  To  make 
matters  worse,  the  answers  may  vary  dramatically  over  time  as  both 
friendly  and  enemy  force  capabilities  evolve. 

MITRE  staff  must  understand  how  the  automatic  tracking  relates 
to  other  system  functions.  Ultimately,  air  defense  gets  down  to 
destroying  enemy  aircraft  before  they  destroy  whatever  is  being 
defended.  One  requirement  for  automatic  tracking  is  that  it  must 
provide  the  location  of  the  enemy  aircraft  with  an  accuracy  sufficient 


to  permit  friendly  weapons  to  kill  the  enemy  target.  But  the  accuracy 
required  may  be  very  different,  depending  on  whether  the  weapon  is 
a  manned  aircraft  or  a  surface-to-air  missile.  And  the  capabilities  of 
the  manned  aircraft  weapon  system  may  vary  widely.  At  one  point  in 
history,  the  computer-generated  position  of  an  enemy  aircraft  needed 
to  be  highly  accurate  to  successfully  guide  the  friendly  weapon  to 
attack  it.  Today,  the  weapon  systems  themselves  have  significantly 
greater  capabilities  to  do  the  weapon  guidance  in  the  final  phase  of 
an  attack.  As  a  result,  the  demands  on  the  accuracy  of  the  tracking 
function  are  significantly  less.  MITRE  staff  must  continually  ask, 
“Why  is  this  function  being  performed?  Under  the  circumstances  that 
exist  in  other  parts  of  the  overall  capability,  what  are  the  perfor¬ 
mance  requirements  on  this  function?” 

But  mitre’s  knowledge  and  understanding  of  air  defense 
extends  far  deeper  than  the  global  issues  or  the  functional  design 
questions  illustrated  above.  At  the  opposite  end  of  the  spectrum, 
the  Corporation’s  work  may  even  demand  that  the  staff  under¬ 
stand  the  meaning  of  the  individual  bits  deep  within  the  computer 
program. 

Again  as  an  example,  at  one  time  the  Air  Force  established  a 
requirement  to  interface  two  systems  that  were  operating  in  Europe 
and  that  had  not  been  designed  to  work  with  one  another — the 
NATO  Air  Defense  Ground  Environment  (NADGE)  system  and  the 
407L  Tactical  Air  Control  System.  NADGE  had  been  acquired  by 
NATO  and  407L  by  ESC.  MITRE  wrote  the  original  specification 
for  NADGE  and  participated  in  its  acquisition.  MITRE  was  also  the 
system  engineer  on  the  407L  program.  The  desired  interface  was  to 
provide  for  computer-to-computer  exchange  of  the  information  on 
the  aircraft  being  tracked  by  each  system.  To  complicate  matters,  the 
using  command  wished  to  have  the  new  interface  capability  opera¬ 
tional  in  less  than  a  year.  In  view  of  the  short  time  available  and  the 
Corporation’s  background  on  both  systems,  it  was  only  natural  for 
the  government  to  come  to  MITRE  for  help  in  interfacing  these 
systems.  The  depth  of  MITRE’s  knowledge  was  illustrated  not  just 
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by  understanding  the  electrical  characteristics  of  the  two  systems  or 
their  different  tracking  logics.  MITRE  staff  was  familiar  with  the 
different  nomenclatures  used  by  the  two  systems  when  referring  to 
different  classes  of  enemy  aircraft  and  even  knew  which  bits  in  the 
computer  messages  were  used  to  convey  those  nomenclatures.  This 
background,  broad  and  deep  in  both  systems,  permitted  the  Corpora¬ 
tion  to  provide  an  initial  version  of  the  necessary  interface  system  in 
a  timely  way  to  meet  user  requirements.  Later,  the  initial  MITRE 
development  models  were  replaced  by  industry  production  systems. 

The  examples  of  the  range  of  MITRE’s  activities  in  an  operational 
mission  area  presented  above  have  concentrated  on  air  defense.  The 
point  of  this  discussion  is  simple  but  profoundly  important:  In  addi¬ 
tion  to  being  technically  competent,  MITRE  staff  in  the  systems 
engineering  role  must,  as  a  group,  have  equal  facility  from  the  bit 
level  in  the  computer  program  to  the  global  issues  that  surround  the 
associated  capability.  When  we  say  MITRE  understands  the  capabil¬ 
ity  that  the  government  wishes  to  achieve,  we  mean  to  cover  this 
entire  range.  MITRE  expertise  of  this  sort  exists  not  only  in  air  de¬ 
fense,  but  also  in  many  operational  areas,  including  other  strategic 
missions,  tactical  operations,  and  intelligence  information  processing. 
Since  these  mission  areas  often  intersect,  the  strength  of  MITRE  and 
its  ability  to  assist  in  achieving  DOD  capabilities  are  reinforced  by 
the  Corporation’s  knowledge  and  experience  across  the  entire  spec¬ 
trum  of  DOD  C^I  systems. 

Assuming  the  staff  has  the  necessary  understanding  of  the  system 
capabilities  needed  by  the  government,  how  is  that  understanding 
applied  in  the  systems  engineering  role?  Successful  completion  of 
complicated  acquisition  programs  requires  continuous  and  careful 
management  by  the  government.  A  program  often  does  not  proceed 
exactly  as  originally  planned.  Difficult  questions  may  arise  at  many 
different  times  during  the  life  of  the  program.  They  will  always  arise 
in  circumstances  where  there  is  a  set  of  apparent  givens — require¬ 
ments  statements,  program  direction,  contracts,  design  decisions, 
resource  commitments,  and  the  like.  They  will  always  occur  in  a 


setting  where  important  people  have  differing  opinions  about  what 
action  to  take,  if  any.  Some  will  prefer  to  wait  and  see  what  happens, 
rather  than  take  an  action  that  later  others  can  say  caused  a  problem 
rather  than  solved  one.  System  performance,  time,  and  money  will 
almost  always  be  factors.  The  preferred  technical  solution  will  not 
match  the  real-world  needs  or  politics.  Egos  and  reputations  may  be 
at  stake.  Presumably,  as  systems  engineer,  MITRE  will  have  an  op¬ 
portunity  to  suggest  appropriate  action.  How  does  the  Corporation 
decide  what  to  recommend.^ 

It  is  assumed  here  that  MITRE  is  a  full  partner  in  the  program  in 
the  way  described  earlier.  The  MITRE  staff  knows  in  a  special  way 
what  the  user  is  trying  to  achieve,  is  fully  informed  on  all  aspects  of 
the  situation,  and  knows  what  is  technically  and  otherwise  feasible. 
No  matter  what  stage  the  program  is  in,  from  early  conception  to 
field  operation,  and  no  matter  the  surrounding  circumstances, 

MITRE  people  must  continually  ask  themselves,  “What  is  the  best 
thing  to  do  at  this  time,  everything  considered,  to  achieve  the  needed 
capability  on  a  reasonable  schedule  and  at  a  reasonable  cost.^” 

This  question,  and  its  answer  in  any  particular  case,  perhaps  more 
accurately  than  anything  else  in  this  book,  describes  the  attitude  and 
contributions  of  MITRE  in  the  systems  engineering  role.  The  Corpora¬ 
tion  is  dedicated  to  helping  the  government  achieve  the  required  capa¬ 
bility,  is  knowledgeable  and  current  on  as  many  of  the  relevant  factors 
as  possible,  and  is  prepared  to  recommend  appropriate  actions. 

The  informed  answer  to  this  question  provides  guidance  during 
normal  program  times.  When  the  situation  on  a  program  gets  truly 
tenuous  and  contentious,  the  knowledgeable  answer  to  this  question  is 
the  only  useful  way  for  the  MITRE  staff  to  decide  what  to  recommend. 
The  program  direction  should  be  considered  in  answering  the  above 
question,  but  if  getting  it  changed  is  the  best  thing  to  do,  MITRE 
should  so  recommend.  The  situation  may  dictate  that  some  compro¬ 
mise  in  the  stated  requirement  is  necessary  to  achieve  a  needed  capabil¬ 
ity.  The  contracts  the  government  has  with  industry  are  important  and 
they  should  not  be  treated  lightly,  but  if  they  need  to  be  changed,  the 
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Corporation  should  recommend  it.  The  argument  about  which  organi¬ 
zation  should  pay  for  the  change,  or  how  much  money  the  government 
should  get  back,  or  whatever,  is  a  separate  one. 

If  change  is  required,  MITRE  should  push  for  it  and  then  help  to 
broker  the  implications  fairly.  The  Corporation  should  be  prepared  in 
such  instances  to  defend  its  recommendations  against  those  who  will 
want  to  take  no  action  until  it  is  almost  too  late,  and  when  the  conse¬ 
quences  of  earlier  inaction  will  be  severe  on  all  those  who  remain. 
MITRE  must  take  the  long  view  but  push  for  early  action.  Most  of  all, 
the  Corporation  must  have  a  good  appreciation  for  what  the  key  play¬ 
ers  in  government  can  tolerate.  There  is  no  sense  asking  people  to  do 
something  they  cannot,  and  no  sense  in  antagonizing  important  people 
without  helping  the  program.  MITRE  must  stand  up  and  be  counted 
even  when  key  people  may  not  like  it,  but  should  do  so  with  full  knowl¬ 
edge  and  with  the  belief  that  it  is  important  to  the  capability  and  within 
the  wherewithal  of  the  people  and  organizations  involved.  MITRE  staff 
must  act  with  the  dignity  and  skill  expected  of  professionals. 

But  how  does  the  Corporation  achieve  the  necessary  understanding 
of  the  government’s  operational  needs  and  capabilities.’  As  men¬ 
tioned,  MITRE’s  35  years  of  work  on  strategic,  tactical,  and  intelli¬ 
gence  systems  have  provided  the  staff  with  a  firsthand  experience 
with  the  various  hardware  and  software  systems  that  are  part  of  the 
various  DOD  mission  capabilities.  The  Corporation  has  had  staff 
working  at  NORAD,  Tactical  Air  Command  Headquarters,  Strategic 
Air  Command,  Air  Force  Logistics  Command,  NATO,  and  many 
other  operating  locations.  The  work  has  helped  these  organizations 
achieve  current  capabilities.  The  experience  gained  is  invaluable  to 
MITRE  in  systems  engineering  for  ESC  of  the  next-generation  CT 
systems.  Through  other  parts  of  the  work  program,  MITRE  staff  is 
familiar  with  the  evolution  of  potential  enemy  systems  to  threaten  or 
defeat  U.S.  C^I  systems.  The  staff  is  also  cognizant  of  enemy  C’l 
systems  in  each  mission  area.  MITRE  project  personnel  use  the 
Corporation’s  unique  across-the-board  knowledge  of  the  mission  area 
to  help  advise  the  U.S.  on  appropriate  countermeasures.  Because  of 


this  collective  experience,  no  other  organization  can  even  begin  to 
match  the  breadth  and  depth  of  MITRE’s  knowledge — in  the  opera¬ 
tional  missions,  in  the  associated  CM  systems,  and  in  the  hardware 
and  software  that  go  to  make  up  these  systems.  It  is  an  invaluable 
resource,  an  unmatched  Air  Force/DOD  corporate  memory  that  is 
both  unique  and  impossible  to  replicate.  This  corporate  memory  is 
supplemented  in  other  ways. 

On  most  programs,  MITRE  participates  in  the  initial  planning  for 
the  capability  and  remains  on  the  program  as  systems  engineer 
through  the  dt'  elopment  phases,  testing,  and  field  operation.  Having 
to  make  a  capability  function  well  in  an  operational  situation  is  both 
a  sobering  and  instructive  experience  for  the  MITRE  staff.  One  learns 
what  is  really  important,  what  works,  and  what  does  not.  The 
uniqueness  of  such  experience  is  commented  on  by  A.D.  Hall; 

It  is  true  that  only  rarely  will  a  particular  person,  or  group,  or 
even  a  whole  organization,  follow  a  system  from  inception 
through  development,  use  and  obsolescence.^^ 

In  MITRE,  corporate  cradle-to-grave  association  with  a  program  is 
more  the  rule  than  the  exception.  It  is  also  true  that  most  CM  systems 
evolve  in  major  ways,  and  the  MITRE  staff  has  had  the  advantage  of 
working  on  the  programs  that  provide  for  additions  and  modifications. 
As  an  example,  the  E-3A  AWACS  for  which  the  Corporation  had 
systems  engineering  responsibility  has  gone  through  a  series  of  major 
upgrades  to  the  system  capability.  In  addition,  versions  of  the  AWACS 
have  been  sold  to  NATO  and  to  Saudi  Arabia.  MITRE  has  had  sys¬ 
tems  engineering  responsibility  on  all  the  upgrades  and  on  all  the 
foreign  sales  programs.  The  continuity  of  the  MITRE  work  program 
ensures  that  the  staff  knowledge  stays  current  as  changes  and  additions 
are  made.  In  turn,  this  avoids  any  lag  in  getting  up  to  speed  in  prepara¬ 
tion  for  the  next  program. 

In  addition  to  DOD  programs,  MITRE  talent  has  also  been  applied 
to  many  other  related  government  programs,  such  as  civil  air  traffic 
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A.D.  Hall,  A  Methodology  for  Systems  Engineering  (Princeton,  New  Jersey: 
D.  Van  Nostrand  Company,  Inc.,  1962),  p.  11. 


control.  The  Corporation  has  built  up  a  large  reservoir  of  talented 
staff  with  firsthand  experience  in  almost  all  the  DOD’s  mission  areas, 
as  well  as  in  other  related  areas. 

But  how  do  individual  staff  members  achieve  this  knowledge, 
understanding,  and  in-depth  appreciation?  Like  so  many  other  skills 
that  take  practice  to  perfect,  the  only  good  way  to  become  a  truly 
outstanding  systems  engineer  is  to  work  as  one.  Skills  in  one  or  more 
technical  areas  are  necessary,  and  a  devotion  to  achieving  a  capability 
for  the  customer  is  essential.  One  way  to  become  expert  at  the  capa¬ 
bility  that  is  to  be  achieved  is  to  work  with  it  from  day  to  day. 
MITRE  has  many  opportunities  for  staff  to  work  at  user  locations 
such  as  Colorado  Springs,  Omaha,  and  Langley.  There  one  may  see 
each  day  what  the  command  personnel  do,  how  they  work,  what  is 
important,  and  what  limitations  exist.  Doing  that  for  a  generation  or 
more  of  user  personnel  begins  to  help  distinguish  what  is  really  im¬ 
portant  and  essential  for  the  successful  operation  of  the  mission 
capability.  There  is  nothing  like  firsthand  observation,  and  if  pos¬ 
sible,  firsthand  operation,  for  the  education  of  MITRE  staff  on  the 
needed  operational  capability. 

Alternatively,  MITRE  personnel  get  many  opporrunifies  to  visit 
operating  locations  to  observe  existing  systems  in  operation  and  to 
discuss  their  strengths  and  weaknesses  with  operating  personnel.  No 
other  corporation  gets  this  opportunity  to  the  same  degree  and  in  so 
many  related  areas.  This  experience  is  less  a  teacher  than  working  at 
an  operating  location  for  an  extended  period,  but  it  is  nonetheless 
valuable. 

Short  of  that  firsthand  experience  at  an  operating  location,  many 
MITRE  projects  provide  a  test  facility  for  the  evaluation  of  existing 
or  planned  capabilities.  In  them,  MITRE  staff  can  get  actual,  first¬ 
hand  experience  at  trying  to  make  the  systems  work.  Such  facilities 
are  not  constructed  for  the  operational  education  of  the  staff;  they 
are  designed  to  answer  specific  questions  about  the  planned  capabili¬ 
ties.  The  more  realistically  they  replicate  the  capability,  the  greater 
their  utility.  Whenever  possible,  they  should  be  staffed  with  people 


MITRE  Joint  STARS  Laboratory 


who  know  the  existing  capability  and  appreciate  what  is  desired  in 
the  new  one,  especially  people  from  the  operating  commands.  How¬ 
ever,  nothing  is  better  for  the  education  of  the  staff  both  on  what  is 
needed  and  what  may  be  possible,  on  what  is  difficult  to  do,  and  how 
to  do  what  needs  to  be  done. 

Another  form  of  the  testbed  is  the  use  of  simulation  facilities. 
Again,  one  can  learn  a  lot,  but  one  can  also  be  deluded.  The  realism 
of  the  simulation  is  critical;  the  assumptions  inherent  in  the  simula¬ 
tion  and  their  validity  must  be  understood.  One  must  be  very  careful 
in  interpreting  the  results  of  a  simulation  in  circumstances  where  the 
inputs  to  the  simulation  only  approximate  those  of  the  live  system,  or 
when  the  simulated  activities  did  not  include  the  influence  of  human 
operators.  Again,  when  used  under  the  tutelage  of  experienced 
people,  simulation  tools  can  answer  many  program  questions  and 
at  the  same  time  contribute  significantly  to  the  growth  of  the  less 
experienced  staff. 

Although  not  directly  related  to  this  discussion  of  how  MITRE  staff 
go  about  becoming  knowledgeable  about  the  capabilities  that  they  are 
trying  to  help  acquire,  it  is  important  to  comment  on  the  use  of  such 
facilities  at  MITRE.  There  is  nothing  about  becoming  a  MITRE  staff 


member  that  automatically  confers  technical  omniscience  on  a  person. 
Some  of  the  programs  on  which  MITRE  has  systems  engineering  respon¬ 
sibility  are  attempting  to  achieve  very  new  and  very  challenging  capabili¬ 
ties.  Doing  so  often  requires  facilities  where  measurements  can  be  made, 
device  performance  e\  aluated,  or  human  interactions  assessed.  Testbeds 
and  simulation  facilities  should  result  only  when  one  is  able  to  pose  the 
questions  that  cannot  otherwise  be  answered,  and  should  cease  to  exist 
when  all  relevant  questions  have  been  resolved.  In  use,  they  may  help 
educate  MITRE  staff.  Elowever,  their  justification  must  be  based  on 
specific  program  needs,  and  their  existence  discussed  with  and  supported 
by  the  system  program  director.  The  use  of  such  facilities  by  MITRE  for 
design  verification  is  discussed  later  in  this  chapter. 

One  can  learn  much  from  talking  to  knowledgeable  people;  how¬ 
ever,  a  person's  organizational  affiliation  does  not  automatically 
make  that  person  knowledgeable.  MITRE  staff  wbo  have  worked  on 
tactical  air  control  systems  for  the  last  30  years  know  a  great  deal 
about  command  and  control  of  tactical  air  forces,  as  well  as  about 
systems  engineering.  A  person  newly  assigned  to  a  tactical  air  unit 
may  or  may  not  know  the  essentials  of  that  mission  area.  Other 
potentially  good  sources  of  personal  contact  include  the  people  work¬ 
ing  on  the  government  side  in  the  SPOs.  Many  of  them  have  had 
experience  operating  in  the  field  or  in  acquiring  earlier  systems  or 
the  operating  commands.  Their  knowledge  and  wisdom  can  be  very 
helpful  to  MITRE  personnel. 

As  one  gets  more  experienced,  one  develops  a  better  personal  sense 
of  what  is  important  and  an  improved  ability  to  recognize  someone 
who  knows  what  they  are  talking  about  from  someone  who  does  not. 
Not  everything  one  will  hear  will  turn  out  to  be  correct.  Until  one 
has  sufficient  experience  to  distinguish  truth  from  fiction,  or  impor¬ 
tant  from  less  so,  one  must  depend  on  the  more  experienced  MITRE 
personnel  for  guidance  and  support. 

Another  way  in  which  MITRE  staff  members  may  obtain  a  deeper 
appreciation  for  the  operational  capability  that  may  be  required  is  to 
attend  one  or  more  of  the  schools  that  the  government  has  and  may 


permit  MITRE  staff  to  attend.  For  example,  many  MITRE  people  have 
attended  the  Air  Force  Air  Ground  Operations  School  in  Florida.  These 
schools  are  designed  to  educate  the  military  personnel  who  will  be 
assigned  to  operate  and  maintain  the  capability  in  question.  They 
provide  much  useful  background  for  the  MITRE  systems  engineer. 

Acquisition  programs  always  include  a  variety  of  test  phases.  There 
are  in-plant  contractor  tests  designed  to  show  that  the  contractor  has 
produced  what  was  promised;  tests  at  operating  locations  to  show  that 
the  delivered  system  meets  its  operational  requirements;  and  tests  of  the 
new  system,  along  with  other  existing  systems,  to  evaluate  the  level  of 
overall  capability  that  has  been  achieved.  MITRE  personnel  often  get  an 
opportunity  to  witness  or  participate  in  such  tests.  A  MITRE  staff 
member  will  gain  precious  knowledge  by  doing  so.  The  depth  of  such 
knowledge  cannot  be  achieved  in  any  other  way.  Following  the  system 
you  have  been  working  on  into  the  field  to  see  how  it  really  operates  is 
sometimes  sobering,  sometimes  exhilarating,  but  always  educational. 
Those  staff  who  do  it  become  much  stronger  system  engineers,  much 
more  able  to  take  on  the  next  challenge  with  wisdom  and  knowledge. 
From  the  very  beginning,  MITRE’s  staff  has  gone  where  the  action  is.  It 
is  part  of  what  makes  MITRE  people  special. 

The  beginning  of  this  section  asserts  that  the  uniqueness  of  The 
MITRE  Corporation  as  a  source  of  systems  engineering  on  Air  Force 
C’l  systems  derived  from  the  corporate  form,  a  deep  appreciation  of 
the  operational  capabilities  involved,  and  over  35  years  of  experience 
working  as  systems  engineer  on  these  systems.  These  points  have  been 
expanded  on  here.  It  is  also  noted  that  the  highly  qualified  technical 
staff  at  MITRE  and  the  dedication  of  the  staff  and  the  management 
to  achieving  required  system  capabilities,  round  out  MITRE’s  unique 
qualifications  as  systems  engineer  for  the  Air  Force’s  most  challenging 
C’l  systems. 

DeaRng  with  Sensitive  Systems  Enghwering  issues 

Over  time,  certain  areas  of  MITRE’s  work  have  proved  to  be 
especially  challenging.  Part  of  the  challenge  has  resulted  from  the 
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sensitivity  or  contentiousness  that  may  surround  the  Corporation’s 
involvement  in  these  areas.  This  section  describes  MITRE’s  approach 
to  a  few  of  the  important  but  potentially  sensitive  areas  of  systems 
engineering. 

AdvMatHKi  System  Copelrilities  Over  the  last  35  years,  there  have  been 
many  different  attitudes  and  regulations  about  the  roles  and  responsi¬ 
bilities  of  the  user  commands  and  the  development  agencies  involved 
in  achieving  the  required  system  capabilities.  Jousting  about  who  is 
responsible  for  what  and  who  is  best  equipped  to  do  what  continues 
even  today.  One  area  that  has  often  come  under  scrutiny  is  the  role 
of  development  personnel  in  advocating  a  system  capability.  At  times, 
the  prevailing  government  acquisition  approach  has  directed  that  the 
development  community  refrain  from  any  form  of  system  advocacy. 

In  Chapter  2,  an  approach  to  system  requirements  is  espoused  in 
which  the  user  community  establishes  the  desired  operational  objec¬ 
tives.  It  is  further  suggested  that  the  user  community  might  establish 
requirements  at  the  level  of  operational  tasks,  but  that  requirements 
for  specific  systems  and  hardware  to  perform  those  operational  tasks 
are  the  business  of  the  development  community.  Clearly,  such  an 
approach  involves  both  the  using  and  developing  communities  in  the 
requirements  business.  There  is  much  room  for  overlap  and  for  de¬ 
bates  about  who  is  responsible  for  what. 

MITRE  fully  understands  and  supports  the  approach  in  which  the 
user  commands — in  conjunction  with  the  service  headquarters,  the 
DOD,  and  ultimately.  Congress,  which  provides  the  resources — 
establish  the  system  requirements  at  the  level  of  operational  objec¬ 
tives  and  tasks.  The  development  commands,  and  others  like  MITRE 
who  work  with  those  commands,  have  key  roles  in  describing  what  is 
possible,  and  how  much  alternative  systems  might  cost.  MITRE 
participates  in  this  work.  When  an  approach  is  chosen,  the  Corpora¬ 
tion  prepares  the  system  specification  enumerating  the  technical 
requirements  that  must  be  satisfied  by  industry  in  building  the  sys¬ 
tem.  In  these  latter  two  areas,  MITRE  contributes  to  the 
government’s  establishment  of  system  requirements. 


MITRE  recognizes  a  responsibility  to  avoid  a  “requirements  creep” 
or  a  “goldplating”  of  requirements  that  could  be  introduced,  for 
example,  by  the  specifications  it  prepares.  Every  effort  is  made  to 
meet  the  stated  system  requirements  without  embellishing  them.  This 
subject  area  has  historically  been  one  of  great  debate,  especially  when 
an  acquisition  program  has  difficulty  in  meeting  the  requirements  of 
the  system  specification.  The  people  in  the  debate  are  often  not  the 
ones  who  approved  the  specification  at  the  time  it  was  written  and 
may  have  different  opinions  about  what  really  should  have  been 
required. 

Despite  risks  of  this  sort,  MITRE  is  dedicated  to  achieving  the  re¬ 
quired  system  capability  as  described  in  the  user  requirements.  In  its 
day-to-day  work,  the  Corporation  is  a  strong  advocate  of  the  capability 
and  is  always  asking  what  needs  to  be  done  to  provide  it.  As  noted 
above,  the  primary  criterion  that  drives  MITRE  in  its  day-to-day  work 
is,  “What  is  the  best  thing  to  do  to  achieve  the  required  capability?” 
Howevet,  if  in  the  cours  of  the  acquisition  program  circumstances 
become  such  that  there  is  significant  risk  that  no  capability  will  be 
achieved,  the  Corpotation  may  recommend  a  change  in  the  require¬ 
ments  to  preserve  essential  portions  of  the  desired  capability.  The  Cor¬ 
poration  has  extensive  experience  with  the  capabilities  under  acquisition 
through  its  work  on  earlier  versions  of  the  system  or  on  like  systems  for 
other  user  commands.  The  nation  has  paid  to  establish  this  knowledge, 
and  it  would  be  wasteful  not  to  apply  it.  MITRE  believes  it  must  advo¬ 
cate  the  capability  in  this  sense  and  that  it  can  do  so  without  usurping 
the  responsibilities  of  the  user  organizations. 

Maintoimiig  System  Design  Integrity  There  is  one  function  that  clearly 
belongs  to  the  system  engineer  and  concerns  the  maintenance  of  the 
integrity  of  the  system  design  as  the  system  proceeds  from  initial 
design,  through  development,  test,  and  field  operation.  One  should  be 
able  to  relate  the  technical  requirements  of  the  system  specification  to 
the  documented  using  command  operational  needs.  This  ensures  that 
the  stated  needs  are  reflected  in  the  technical  specification.  It  also 
helps  to  avoid  including  extraneous  technical  requirements  that  could 
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be  costly  and  time-consuming.  The  MlTRE-prepared  system  specifica¬ 
tion  is  used  by  the  government  in  the  process  of  contractor  selection. 

Later  in  an  acquisition  program,  traceability  to  the  MITRE  system 
specification  should  be  a  criterion  for  evaluating  the  contractor  speci¬ 
fication.  A  similar  chain  of  logic  is  followed  by  the  Corporation  in 
assessing  other  documentation,  such  as  proposed  test  plans.  The 
question  is  always  of  a  similar  form:  “Is  what  is  contained  here 
consistent  with  the  system  design?  Will  the  test  verify  consistency 
with  the  system  design?”  In  that  way,  MITRE  is  in  effect  asking, 

“Will  this  contribute  to  achieving  the  required  capability?” 

The  system  specification  acts  as  a  standard  to  be  used  by  the 
Corporation  in  evaluating  events  that  take  place  on  the  program.  As 
has  been  discussed,  dynamic  change  is  a  way  of  life  on  a  major  acqui¬ 
sition  program.  As  other  events  occur,  both  within  the  program  itself 
and  in  other  related  programs,  MITRE  evaluates  them  for  impact  on 
the  system  design.  Changes  in  requirements  may  affect  system  design. 
Similarly,  changes  in  areas  such  as  technology  or  threat,  or  develop¬ 
ment  breakthroughs  or  failures,  may  require  system  design  changes. 

In  any  case,  MITRE  as  part  of  its  evaluation  of  the  current  situation 
is  always  examining  the  impact  on  the  system  design.  Retaining  the 
existing  design  may  be  the  best  thing  to  do  to  achieve  a  capability. 
The  Corporation  must  be  prepared  to  recommend  that  and  to  identify 
any  actions  required  to  do  so.  Alternatively,  circumstances  may  be 
such  that  MITRE  will  recommend  that  the  system  design  be  modified 
to  obtain  the  operational  capability.  Again,  the  impact  of  doing  so 
must  be  described.  MITRE  must  be  pragmatic  in  its  efforts  and  know 
when  compromise  is  necessary.  At  the  same  time,  MITRE  must  be 
prepared  to  recommend  what  is  necessary  to  preserve  the  design  and 
thereby  ensure  the  capability. 

The  MITRE  staff  has  extensive  educational  backgrounds  that  help 
provide  a  foundation  for  the  technical  requirements  written  in  a 
system  specification.  The  staff  is  similarly  steeped  in  operational 
considerations  through  the  Corporation’s  work  on  large  military 
systems  over  the  last  35  years.  With  the  wisdom  gained  from  experi- 


ence  and  current  knowledge  of  available  technology,  staff  members 
are  well  prepared  to  write  specifications  for  new  system  capabilities 
that  may  be  required  by  the  user  community.  However,  this  does  not 
mean  that  in  every  case  the  staff  can  do  so  without  investigating  some 
of  the  potential  unknowns  or  without  evaluating  areas  of  high  risk.  It 
is  further  true  that  such  investigations  and  evaluations  are  often  in 
very  complex  technical  and  operational  areas  and  are  not  amenable 
to  mere  pencil  and  paper  analysis. 

Parforariig  Dtsigi  VwHkatiM  Historically,  MITRE’s  work  to  resolve 
unknowns  or  evaluate  risks  on  an  acquisition  program  has  been 
referred  to  as  design  verification.  Design  verification  may  start  very 
early  in  the  process,  even  before  the  overall  system  design  is  com¬ 
plete.  It  continues  throughout  the  program.  As  noted  in  a  MITRE 
paper,  its  objectives  may  include  establishing  a  workable  design, 
choosing  among  competitive  designs,  optimization  of  a  particular 
design,  or  early  identification  of  unforeseen  problems.^^  The  methods 
of  design  verification  may  include  one  or  more  of  the  following: 
theoretical  studies,  modeling,  simulation,  mockups,  engineering  mod¬ 
els,  and  prototypes.  It  should  be  pointed  out  that  the  MITRE  corpo¬ 
rate  memory  carries  over  from  program  to  program  and  helps  to 
minimize  the  need  for  design  verification. 

MITRE’s  design  verification  activities  provide  information  to  ail 
program  participants,  not  just  the  Corporation.  Even  simple  efforts 
such  as  model  shop  versions  of  operator  consoles,  or  prototyping  of 
electronic  displays,  are  very  helpful  in  assisting  user  personnel  to 
visualize  the  operator  facilities.  This  visualization  provides  a  good 
basis  for  user  personnel  comments  on  the  planned  system  design.  The 
early  understanding  of  what  is  involved  may  also  help  the  user  com¬ 
mand  in  planning  for  personnel  staffing  and  training. 

At  the  opposite  end  of  the  spectrum,  if  the  government  is  assessing 
how  to  best  spend  its  limited  resources  to  improve  air  defense  in 
some  operating  area  of  the  world,  the  MITRE  answer  may  require 
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considerable  study  and  simulation.  Such  studies  would  assess  the 
current  capability  and  the  contribution  that  might  be  made  to  it  by 
alternatives,  such  as  a  major  new  radar  system,  or  expenditures  for 
higher  capacity  or  more  survivable  command  and  control  centers,  or 
the  acquisition  of  a  new  and  more  capable  weapons  system.  Govern¬ 
ment  decision  makers,  in  turn,  can  also  use  the  analysis  data  that 
would  not  be  otherwise  available  to  choose  among  the  possible  alter¬ 
natives.  Such  a  study  could  not  be  done  by  profit-making  industry 
without  potential  for  conflict  of  interest,  and  there  are  no  alternative 
sources  with  MITRE’s  long  and  deep  experience  in  areas  such  as  air 
defense  and  other  military  mission  areas  of  which  C^I  systems  are 
such  an  integral  part. 

Design  verification  requires  facilities  and  personnel  resources.  No 
one  should  expect  MITRE  staff  to  have  all  the  answers  without 
investments  of  the  sort  common  to  all  professionals.  One  would  not 
think  of  pursuing  a  new  airplane  program  without  many  different 
analyses  and  wind  tunnel  tests.  Similarly,  resolving  some  of  the  un¬ 
knowns,  or  evaluating  some  of  the  risks  on  challenging  CM  acquisi¬ 
tion  programs,  also  require  design  verification  activities.  The 
Corporation  must  identify  areas  that  clearly  require  MITRE  system 
design  verification  work  and  vigilantly  pursue  the  resources  necessary 
to  do  it.  Alternatively,  special  studies  that  are  best  accomplished  by 
other  program  participants  should  be  recommended  by  MITRE  and 
pursued  until  they  are  successfully  accomplished. 

What  is  important  here  are  the  questions  to  be  answered,  the 
unknowns  to  be  explored,  the  risks  to  be  assessed  and  reduced,  as 
they  affect  the  system  program  for  which  MITRE  has  the  system 
engineering  role.  Too  often,  one  tends  to  talk  about  the  facilities  and 
personnel  required  to  do  the  work  and  not  enough  about  the  ques¬ 
tions  to  be  answered  and  their  importance  to  a  successful  acquisition 
program.  This  latter  approach  may  inadvertently  create  tension  be¬ 
tween  MITRE  and  the  SPO  and  rejection  of  the  MITRE  initiative, 
as  though  the  proposed  effort  had  no  bearing  on  the  acquisition 
program. 


Clearly,  a  program  director  has  very  limited  resources  and  many 
areas  in  which  to  use  them.  The  number  of  MITRE  staff  available  to 
support  the  program  is  always  constrained,  and  the  jobs  for  them  to 
do  are  many.  It  is  quite  understandable  that  a  program  director 
would  hope  that  MITRE  staff  knows  all  the  necessary  answers 
without  doing  special  studies  and  experiments  that  can  consume 
significant  resources.  It  is  equally  clear  that  a  program  director  will 
want  to  be  sure  that  an  investment  in  MITRE  design  verification 
work  is  in  the  best  interest  of  the  overall  program.  The  Corporation 
must  be  careful  in  recommending  and  describing  such  activities  to  be 
sure  they  are  necessary  to  meet  its  obligations  under  the  systems 
engineering  role.  Similarly,  the  program  director  must  provide  the 
necessary  MITRE  staff  years  of  effort  and  the  facilities  necessary  to 
perform  approved  design  verification  activities.  When  the  questions 
have  been  answered  and  the  risks  assessed,  these  facilities  and  the 
personnel  who  use  them  should  be  redirected  either  to  other  work  on 
that  program  or  to  other  programs. 

Design  verification  activities,  matched  to  the  needs  of  the  indi¬ 
vidual  system  acquisition  program,  are  an  essential  ingredient  of 
MITRE’S  systems  engineering  work.  In  discussing  the  need,  the  Cor¬ 
poration  must  be  prepared  to  describe  in  detail  what  unknowns  need 
to  be  resolved,  what  risks  require  investigation,  and  why.  Then,  an 
approach  to  doing  so  must  be  proposed.  Finally,  the  resources  re¬ 
quired  must  be  described.  It  is  only  after  the  need  for  design  verifica¬ 
tion  is  agreed  upon  that  discussing  the  facilities  to  do  so  makes  any 
sense.  In  that  way,  the  connotation  that  such  activities  are  an  unnec¬ 
essary  cost  of  doing  business  with  MITRE  will  be  eliminated.  Design 
verification  is  a  crucial  systems  engineering  function,  no  matter  which 
organization  performs  in  that  role. 

Asstssiig  ProfrdM  Chwys  The  acquisition  programs  that  provide  the 
systems  necessary  to  achieve  important  government  capabilities  are 
large,  complicated,  very  costly,  and  take  years  to  complete.  Many 
government  agencies  and  industrial  concerns  are  involved.  Each 
program  is  conducted  in  a  competition  for  resources  with  the  other 
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funding  demands  placed  on  the  government.  The  competition  for 
funds  continues  annually  throughout  the  life  of  the  program,  and  the 
competitors  are  constantly  changing.  It  is  not  difficult  to  imagine  in 
such  circumstances  that  the  program  will  be  subject  to  many  changes 
both  from  within  and  without. 

One  of  the  major  functions  of  MITRE  as  system  engineer  for  many  of 
these  programs  is  to  help  the  government  anticipate  the  need  for  change 
and  make  those  changes  deemed  necessary  at  any  time  in  the  program. 
Changes  in  the  planned  acquisition  program  may  be  necessary  to  react 
to  a  new  enemy  threat,  adapt  to  new  information  on  the  available 
technology,  accommodate  modified  user  requirements,  or  compensate 
for  failures  in  some  portion  of  the  planned  acquisition.  Modification  of 
the  program  may  be  dictated  by  things  that  happen  in  other  related 
programs  that  are  contributing  to  the  mission  capability.  Changes  in  the 
available  money  or  other  resources  will  often  require  modifications  to 
an  acquisition  program.  Change  management  is  a  critical  function  of  the 
system  program  management  office.  As  system  engineer,  MITRE  plays  a 
special  role  in  helping  the  SPO  perform  that  function. 

Before  discussing  MITRE’s  participation  in  the  change  control 
process  on  a  program,  there  are  two  cautionary  observations  that 
need  to  be  made.  As  mentioned  so  often  in  this  book,  the  MITRE 
criteria  for  evaluating  the  need  for  a  change  or  the  specifics  of  any 
proposed  change,  are  based  on  what  is  the  best  thing  to  do  at  this 
time  to  achieve  the  requirea  capability  on  a  reasonable  schedule  and 
at  a  reasonable  cost.  As  discussed  below,  this  evaluation  will  often 
have  to  be  done  when  there  is  less  data  than  one  would  like,  and 
when  different  program  participants  have  strongly  varying  opinions 
on  how  to  proceed.  Often,  MITRE  will  have  to  make  judgments 
based  on  the  professional  experience  of  the  staff.  The  Corporation 
must  be  prepared  to  do  so  and  to  strongly  recommend  appropriate 
action.  MITRE’s  success  in  a  given  instance  will  be  determined,  in 
part,  by  the  strength  of  the  relationships  the  Corporation  has  built 
during  the  program,  and  particularly  by  the  relationship  between  the 
government  program  director  and  the  MITRE  project  leader. 


The  second  cautionary  note  should  be  a  little  more  obvious.  MITRE  is 
not  in  the  business  of  crying  wolf  all  day,  every  day.  As  noted  above,  the 
Corporation  must  be  correct  in  its  professional  assessments  and  must  also 
be  careful  not  to  bog  down  the  system  with  trivia.  One  must  distinguish 
those  actions  that  are  truly  significant  to  the  achievement  of  the  capability 
from  those  that  one  might  refer  to  as  neatness.  A  faulty  design  may  criti¬ 
cally  affect  the  capability,  a  misspelling  in  a  specification  may  merely  be  a 
problem  in  nearness.  MITRE  must  carefully  distinguish  those  maners  that 
are  critical  to  achieving  the  required  capability.  As  a  corollary,  the  Corpo¬ 
ration  also  needs  to  give  credit  when  credit  is  due.  When  one  of  the  pro¬ 
gram  participants  has  done  a  good  job,  MITRE  should  stand  up  and  say 
so.  If  a  specification  is  good  for  the  most  part  but  has  a  few  problems,  the 
Corporation’s  comments  should  include  recognition  of  the  good,  along 
with  description  of  what  needs  to  be  changed.  In  other  words,  MITRE 
needs  to  be  both  objective  and  professional. 

It  should  be  noted  that  the  need  to  make  a  change  can  often  be  viewed 
in  two  different  ways.  Some  people  will  look  at  the  need  for  change  as  a 
problem — more  work,  or  more  money,  or  a  sign  of  failure.  Others  will 
look  at  change  as  an  opportunity — to  increase  the  chances  of  success, 
improve  the  capability  that  will  be  delivered,  or  eliminate  some  problem 
that  existed.  To  cite  one  example.  Congress  at  one  time  threatened  to 
stop  funding  the  AWACS  program.  Some  members  of  the  Congress  felt 
that  since  AWACS  was  intended  for  continental  air  defense,  and  since 
that  threat  was  no  longer  significant,  the  program  should  be  canceled. 
That  certainly  created  an  unanticipated  problem  for  the  program  and  for 
anyone  who  believed  the  AWACS  represented  an  important  national 
capability.  However,  thoughtful  people  in  MITRE  and  elsewhere  had 
always  viewed  AWACS  as  a  system  with  potential  utility  in  a  wide 
variety  of  mission  areas,  not  just  continental  air  defense.  Many  felt 
AWACS  could  make  a  valuable  contribution  to  tactical  command  and 
control.  To  examine  that  possibility,  the  Air  Force  decided  to  demon¬ 
strate  the  system’s  capability  in  Europe.  MITRE  played  a  major  role  in 
that  demonstration,  and  the  flight  tests  were  successful  in  helping  impor¬ 
tant  people  to  realize  the  multimission  capabilities  of  AWACS.  Funding 


In  every  problem,  there  is 
opportunity;  one  has 
only  to  find  it. 


NATO  Secretary  (leneral  Luns  at  AWACS  European  Demonstration 


was  quickly  restored  and  the  capability  of  the  resulting  system  has  been 
repeatedly  demonstrated  throughout  the  world  in  a  variety  of  operational 
situations  over  the  last  several  years. 

The  decision  to  provide  an  early  demonstration  of  AWACS  system 
capability  in  Europe,  for  everyone  to  see,  was  a  challenge.  It  had  not 
been  planned.  New  effort  and  resources  were  necessary,  and  there 
was  significant  risk  of  failure.  However,  the  Air  Force,  and  especially 
the  system  program  office  at  ESC,  MITRE,  and  the  system  contrac¬ 
tor,  Boeing  Airplane  Company,  all  viewed  the  European  flights  as  an 
opportunity  to  demonstrate  what  they  thought  was  an  important 
mission  capability.  More  than  that,  MITRE  suggested  that  early 
MITRE-built,  advanced  development  data  link  terminals  should  be 
used  to  distribute  the  AWACS  data  in  real  time  from  the  aircraft  to  a 
variety  of  ground  and  seaborne  installations  throughout  Europe.  In 
that  way,  people  on  the  ground  could  see  for  themselves  what  infor¬ 
mation  the  AWACS  could  provide  and  how  it  might  be  distributed  to 
friendly  ground  command  and  control  facilities.  The  recommendation 
was  accepted  and  resulted  in  the  first  demonstration  of  what  is  now 
known  as  JTIDS.  MITRE  staff  participated  directly  in  the  successful 
flight  tests.  The  demonstrations  not  only  helped  restore  AWACS 
funding,  but  through  the  initiative  takei,  by  MITRE,  they  gave  major 
impetus  to  the  fledging  JTIDS  program.  In  every  problem,  there  is 
opportunity;  one  has  only  to  find  it. 
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The  anticipation  and  accommodation  of  the  future  need  for  change 
is  one  part  of  MITRE’s  systems  engineering  role.  In  planning  for  a 
mission  capability,  one  finds  that  the  capability  will  evolve  over  time 
as  new  subsystems  become  available.  The  command  and  control 
facilities  will  have  to  process  data  from  new  sensors  or  will  have  to 
exchange  data  with  new  weapon  platforms  that  place  new  demands 
on  the  control  system.  Since  each  of  the  acquisition  programs  is  being 
carried  out  on  its  own  schedule,  and  since  the  command  and  control 
facilities  must  interact  with  all  of  them,  design  of  the  C^l  systems 
must  account  for  this  evolution.  Evolutionary  development  of  the 
command  and  control  facility  may  be  dictated.  A  plan  needs  to  be 
developed  in  which  the  system  is  delivered  in  increments  designed  to 
match  the  systems  being  provided  by  other  SPOs.  To  facilitate  imple¬ 
mentation  of  the  plan,  the  MITRE-prepared  C^I  system  specification 
may  require  the  use  of  design  approaches  that  facilitate  adding  system 
capacity  or  special  features  as  the  need  arises.  Modular  design  of  the 
hardware  and  software,  or  the  use  of  bus  communications  for  inter¬ 
connecting  the  modules,  are  two  such  possibilities. 

MITRE’s  anticipation  of  future  needs  cannot  be  limited  to  system 
design  considerations.  Perhaps  special  equipment  will  be  required  to 
test  the  new  capability,  but  not  to  operate  it.  The  Corporation 
should  recognize  such  a  need  early  in  the  program  and  work  with 
the  SPO  to  ensure  the  test  equipment  is  available  to  match  the  test 
schedule.  If  not  recognized  in  a  timely  way,  such  needs  can  cause 
significant  delays  in  the  program  schedule  and  may  result  in  major 
cost  increases  as  the  system  waits  until  the  necessary  test  equipment 
is  made  available.  MITRE  staff  members  need  to  continually  think 
ahead  in  all  areas  to  anticipate  what  needs  to  be  done  to  increase  the 
probability  of  achieving  the  required  capability  in  a  timely  and  cost- 
effective  manner. 

Most  of  the  changes  that  take  place  over  the  life  of  an  acquisition 
program  are  not  specifically  predictable.  One  might  foresee  that 
resources  for  strategic  defense  will  be  shrinking  over  the  next  10 
years,  without  being  able  to  foretell  whether  the  cuts  will  take  place 


in  force  structure,  development  funds,  weapons,  or  command  and 
control  capabilities.  Such  cuts  might  take  place  when  no  reasonable 
person  would  have  predicted  them.  Few  people  foresaw  the  rapid 
pace  of  change  in  U.S./Soviet  relations,  and  therefore  few  anticipated 
the  obvious  impacts  on  defense  programs.  Indeed,  most  of  the 
changes  that  must  be  accommodated  in  an  acquisition  program  are 
not  ones  that  can  be  specifically  planned  for  ahead  of  time.  They 
often  represent  difficult  choices  that  must  be  made  in  a  timely  way  to 
minimize  their  impact  on  system  performance,  schedule,  and  cost. 

It  is  also  true  that  most  of  the  unanticipated  change  that  must  take 
place  in  a  system  acquisition  program  does  not  stem  from  global 
issues  such  as  congressional  funding  cuts  or  improved  U.S./Soviet 
relationships.  Events  such  as  these  are  of  great  significance,  but  they 
are  few  and  get  everyone’s  attention.  Indeed,  most  of  the  need  for 
change  is  the  result  of  a  variety  of  things  that  always  take  place 
within  a  given  acquisition  program  or  in  a  related  program  with 
which  it  must  interface  to  provide  the  required  mission  capability.  A 
piece  of  the  system  will  not  quite  meet  its  performance  requirements 
and  a  decision  will  have  to  be  made  either  to  fix  it,  or  to  modify 
some  other  part  of  the  system  to  compensate,  or  perhaps  to  live  with 
the  reduced  capability.  An  activity  will  not  be  completed  on  time  and 
other  activities  will  potentially  be  affected.  Work-arounds  will  have 
to  be  found  to  minimize  lost  time  and  resources.  New  people  will  be 
assigned  to  the  program  in  government  or  industry,  and  they  may 
have  different  opinions  about  some  of  the  details.  As  noted  in  Chap¬ 
ter  3,  the  environment  within  which  these  programs  are  conducted  is 
incredibly  complicated.  Managing  change  is  one  of  the  most  challeng¬ 
ing  tasks  facing  the  government  program  director.  As  systems  engi¬ 
neer,  one  of  MITRE’s  most  important  efforts  is  to  help  the  program 
director  do  that  in  ways  that  ensure  the  needed  capability  is  achieved 
on  a  reasonable  schedule  and  at  a  reasonable  cost. 

To  do  that,  MITRE  must  be  aware  of  what  is  happening  in  the 
environment  within  which  the  program  takes  place.  The  Corporation 
must  also  understand  firsthand  the  status  of  all  activities  on  the  pro- 


gram  itself.  Staff  must  study  potential  risk  areas,  perform  the  necessary 
design  verifications,  read  the  associated  documents,  attend  the  various 
tests  and  analyze  the  results,  participate  in  program  reviews,  assess 
progress,  and  become  familiar  with  all  relevant  activities.  MITRE  must 
take  the  initiative  to  identify  opportunities  and  problems,  and  to  recom¬ 
mend  actions  to  exploit  the  opportunities  or  to  overcome  the  problems. 
The  Corporation  must  prepare  alternative  courses  of  action,  evaluate 
them,  and  provide  the  information  to  government  decision  makers. 
MITRE  staff  must  be  proactive  in  this  regard.  However,  a  well-in¬ 
formed  MITRE  staff  is  also  prepared  to  advise  the  system  program 
director  on  recommendations  made  by  other  program  participants.  In 
consonance  with  the  program  director,  MITRE  must  work  for  changes 
necessary  to  achieve  the  required  capability,  and  against  those  that  are 
unnecessary  or  detrimental  to  that  objective. 

In  helping  to  manage  change,  the  staff  must  recognize  that  within 
each  large  program,  immersed  as  it  is  in  its  universe,  there  are  many 
simultaneous  activities  that  could  dictate  change  at  any  given  time. 
The  challenge  is  to  recognize  those  that  are  important  and  deal  with 
them  swiftly.  Every  day  there  will  be  more  happening  on  a  program 
than  can  possibly  be  absorbed  and  responded  to  in  real  time.  That 
problem  will  face  MITRE  and  the  system  program  director,  as  well. 
The  Corporation  must  work  to  eliminate  the  trivia  from  consuming 
program  resources  and  direct  attention  to  the  most  important  mat¬ 
ters,  while  at  the  same  time  helping  the  program  director  deal  with 
other  necessary  activities.  Program  directors  are  often  required  to 
carry  out  effort  only  remotely  related  to  achieving  the  capability. 
Briefing  visiting  dignitaries  is  one  mundane  example.  Such  activities 
take  time  and  resources  away  from  the  program,  but  they  are  im¬ 
portant  because  the  program  director  has  to  see  that  they  are  done, 
and  done  well.  Since  MITRE  is  in  the  business  of  helping  the 
program  director,  the  Corporation  must  be  prepared  to  provide 
professional  support  to  them.  With  limited  resources,  achieving 
appropriate  balance  among  these  demands,  while  ensuring  that 
MITRE’S  direct  responsibilities  are  effectively  met,  is  the  daily 
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challenge  faced  by  the  MITRE  project  leader.  The  quality  of  the 
MlTRE/program  director  relationship  will  be  a  function  of  how 
well  the  project  leader  makes  the  necessary  choices.  This  relation¬ 
ship,  in  turn,  will  determine  how  effective  the  Corporation  is  in 
helping  to  achieve  the  required  capabili.v. 

There  is  another  aspect  of  the  challenge  to  MITRE  in  helping  to 
manage  change  that  deserves  discussion.  Obviously,  the  timeliness  of 
mitre’s  advice  is  very  important.  In  most  cases,  the  earlier  one 
recognizes  the  need  for  a  change,  the  less  the  change  will  cost,  the 
more  readily  it  will  be  accepted  by  those  who  must  implement  it,  and 
the  earlier  the  capability  will  be  achieved.  A  technical  recommenda¬ 
tion  made  early  in  the  contractor  design  phase  may  be  easily  adopted 
by  the  contractor.  No  great  investment  has  been  made  in  the  design. 
No  one’s  professional  reputation  is  on  the  line,  so  the  not-invented- 
here  syndrome  is  not  likely  to  -e  a  factor.  The  same  technical  recom 
mendation  at  time  of  production  might  be  impossible.  MITRE  must 
always  be  looking  ahead,  trying  to  discern  actions  that  if  taken  now 
will  be  important  later.  On  the  other  hand,  there  are  factors  that 
mitigate  against  being  able  to  make  early  changes. 

MITRE  is  often  in  a  situation  where  it  suspects  a  serious  problem 
is  developing,  but  is  unable  to  prove  it,  and  no  other  participant  is 
willing  or  able  to  corroborate  it.  Or  perhaps  other  participants  are 
optimistic  about  their  ability  to  deal  with  it.  The  Corporation’s 
professional  experience  indicates  that  the  problem  is  real,  but  the 
evidence  is  circumstantial.  Contrary  to  what  was  said  above,  such 
problems  are  very  often  difficult  to  resolve  as  early  as  would  be  desir¬ 
able.  For  such  problems,  the  earlier  one  tries  to  recommend  change, 
the  more  difficult  it  is  to  persuade  people  to  make  it.  Change  often 
means  additional  effort  or  resources — negotiation  with  other  partici¬ 
pants,  contract  changes,  more  money,  etc.  There  is  a  natural  institu¬ 
tional  resistance  to  significant  change  at  any  time,  not  unlike  that 
found  in  most  individuals.  This  resistance  must  be  overcome  ii  a  timely 
change  is  important  for  achieving  the  capability.  In  situations  like 
this,  when  the  matter  is  truly  significant  to  achieving  the  capability. 


MlTRE’s  relationship  with  the  program  director  comes  into  play.  If 
that  relationship  is  as  described  earlier  in  this  book,  then  MITRE  will 
most  often  be  successful  in  convincing  the  program  director  of  a  cru¬ 
cial  need  for  change.  This  is  true  even  if  the  change  is  contentious  and 
the  evidence  for  it  less  than  absolute. 

Another  alternative  for  dealing  with  risk  areas  or  suspected  prob¬ 
lems  is  to  take  some  form  of  insurance  against  the  occurrence  of  a 
later  problem.  MITRE  should  consider  this  approach  in  attempting 
to  deal  with  known  or  suspected  risk  areas.  When  necessary,  the 
staff  should  take  the  initiative  to  identify  specific  actions  that 
should  be  taken  and  work  as  part  of  the  program  team  to  imple¬ 
ment  them.  Everyone  wants  to  believe  that  things  will  go  as 
planned.  No  one  wants  to  hear  bad  news.  Most  especially,  people 
who  have  to  Jo  something  that  is  costly  or  unpleasant  as  a  result  of 
the  bad  news  like  it  even  less,  and  they  have  a  tendency  not  to  hear. 
Telling  people  what  they  need  to  hear  is  never  an  easy  task,  but 
MITRE  is  paid  to  do  just  that.  Sometimes  the  chances  of  things 
working  out  as  planned  are  sufficiently  small  that  the  plan  to 
achieve  the  required  capability  on  time  and  within  cost  is  in  serious 
jeopardy.  MITRE  must  be  sensitive  to  such  situations  and  take  the 
initiatives  deemed  necessary  to  reduce  the  risk  to  an  acceptable 
level.  Again,  it  is  human  to  hope  for  the  best.  With  limited  re¬ 
sources,  one  tries  to  avoid  using  them  to  hedge  against  future  pos¬ 
sible  risk.  But  sometimes  professional  experience  dictates  that  action 
is  required  to  avoid  serious  problems  later  in  the  program.  Such 
occasions  arise  in  every  profession.  To  reiterate  a  point  made  ear¬ 
lier,  MITRE  must  be  prepared  to  give  the  program  director  the 
benefit  of  the  professional  experience  of  its  staff.  Again,  the 
Corporation’s  prior  work  on  the  program  will  heavily  influence  the 
program  director’s  response. 

As  noted  above,  some  changes  are  readily  accepted  if  made  early  in 
the  program.  A  change  in  the  design  phase  is  more  readily  made  than 
after  the  first  units  have  been  produced.  Software  changes  in  the  cod¬ 
ing  phase  are  much  more  readily  done  than  after  the  system  becomes 


operational.  For  such  matters,  as  the  program  progresses  in  time,  it 
gets  increasingly  difficult  to  convince  people  that  a  change  is  required. 
Resources  have  been  invested  in  the  current  approach,  commitments 
have  been  made  to  important  people,  no  one  likes  to  admit  to  being 
wrong,  and  in  some  cases  the  resources  most  capable  of  making  the 
change  are  no  longer  available.  MITRE  must  evaluate  the  need  for  the 
change  and  the  manner  in  which  it  might  be  done.  The  Corporation 
should  then  work  as  part  of  the  program  team  in  getting  the  necessary 
changes  approved  and  implemented. 

Obviously,  making  changes  is  costly  and  time-consuming.  Ques¬ 
tions  such  as  who  should  pay  for  the  change  always  arise.  The 
answers  are  often  legitimately  argumentative.  Usually,  all  the  parties 
involved  have  contributed  to  the  situation,  and  all  have  to  share  in 
its  resolution.  It  is  always  helpful  to  try  to  separate  the  concern  for 
who  pays  from  the  nature  of  the  change  itself,  although  that  is  not 
easy  to  do.  If  such  an  approach  is  used  by  MITRE,  what  is  best  for 
achieving  the  capability  becomes  the  paramount  consideration, 
rather  than  who  is  at  fault  for  the  problem  and  therefore  who 
should  pay.  When  the  stakes  are  high,  some  parties  will  evaluate  the 
situation  first  on  the  basis  of  who  is  liable,  and  only  secondarily  on 
what  is  best  to  achieve  the  capability.  The  Corporation  needs  to  be 
prepared  to  deal  with  that  situation.  In  other  cases,  who  will  have 
to  pay  can  become  more  of  a  factor  for  everyone.  Eor  example, 
when  there  are  a  variety  of  possible  solutions  that  vary  in  who  will 
provide  them,  the  evaluation  provided  by  MITRE  needs  to  specifi¬ 
cally  include  this  factor. 

Judging  which  changes  are  absolutely  necessary  and  when  they 
should  be  made,  is  a  very  difficult  management  task.  Helping  the  SPO 
to  do  that  is  a  major  MITRE  systems  engineering  effort.  Doing  it  well 
requires  a  professional  attitude,  skill,  and  experience.  It  demands  a 
dedication  to  the  required  capability,  thorough  understanding  of  the 
status  of  all  program  activities  at  all  times,  and  willingness  to  say  what 
needs  to  be  said.  It  also  requires  a  relationship  with  the  program  direc¬ 
tor  in  which  he  or  she  relies  on  MITRE  as  a  tull  program  partner. 


Applying  Systm  AcqiiisHiM  RagnlatiMS  One  of  the  imposing  and  vexing 
challenges  faced  by  MITRE  in  the  systems  engineering  role,  and 
indeed  by  all  participants  in  a  major  DOD  acquisition  program,  is  the 
vast  array  of  government  documentation  that  controls  or  potentially 
applies  to  such  procurements.  As  was  observed  earlier,  no  regulation 
can  substitute  for  knowledgeable  management  in  ensuring  a  success¬ 
ful  program.  However,  some  regulations  are  legally  binding  on  the 
acquisition  process,  and  some  provide  good  guidance  if  appropriately 
employed.  Many  permit  selective  application,  as  may  be  necessary  in 
any  particular  acquisition  program.  The  imposition  of  some  regula¬ 
tions  may  help  or  hinder  the  success  of  a  program.  Many  government 
agencies  are  involved  in  deciding  which  regulations  apply  and  how. 

As  systems  engineer,  MITRE  is  also  involved  in  the  proper  applica¬ 
tion  of  some  of  this  documentation.  The  overlap  between  MITRE  and 
government  people  can  be  a  positive  influence  on  the  program  when 
the  expertise  and  experience  of  the  two  groups  are  applied  coopera¬ 
tively.  It  may  also  be  an  area  of  serious  contention  when  one  group 
or  the  other  is  unwilling  or  unable  to  take  an  approach  in  which  the 
regulations  are  used  to  assist  the  program,  rather  than  unnecessarily 
burden  it  with  requirements  just  to  be  on  the  safe  side.  Unnecessary 
requirements — especially  in  areas  such  as  testing  and  documenta¬ 
tion — can  be  an  extremely  expensive  and  frustrating  process  in  time 
and  resources  for  both  the  government  and  industry. 

In-depth  discussion  of  the  government’s  system  acquisition  regula¬ 
tions  is  beyond  the  purpose  of  this  paper.  However,  their  scope  will 
be  briefly  described  here.  Some  suggestions  will  be  provided  on  how 
MITRE  should  participate  in  tailoring  the  required  regulations  to 
specific  acquisition  programs. 

Some  of  the  high  level  documents  that  have  governed  DOD  acqui¬ 
sition  programs  for  some  time  include: 

•  Office  of  Management  and  Budget  Circular  A- 109,  Major 
Systems  Acquisitions,  April  5,  1976 

•  The  Federal  Acquisition  Regulation  (FAR)  Series 

•  Defense  Federal  Acquisition  Regulation  Supplements 


Unnecessoiy  requirements — 
especiolly  in  areas  such  os 
testing  and  documentation — 
con  be  an  extremely  expensive 
ond  frustrating  process  in  time 
and  resources  for  both  the 
government  ond  industry. 


More  recently,  a  formidable  series  of  legislative  constraints  (Title 
10,  United  States  Code)  has  been  imposed.  The  sections  of  this  code 
cover  subjects  such  as  competitive  prototyping,  operational  testing 
and  evaluation,  low-rate  initial  production  of  new  systems,  and  the 
use  of  competitive  alternative  sources  on  major  programs.  This  legis¬ 
lation  has  led  to  a  revision  (23  February  1991)  of  DOD’s  system 
acquisition  regulations  as  found  in: 

•  DOD  Directive  5000.1,  Defense  Acquisition 

•  DOD  Instructions  5000.2,  Defense  Acquisition  Management 
Policies  and  Procedures 

•  DOD  Manual  5000.2-M,  Defense  Acquisition  Management 
Documentation  and  Reports 

These  documents  describe  DOD’s  approach  to  implementing  the 
U.S.  Code  and  FAR  mandates.  They  detail  the  DOD’s  three  major 
decision-making  management  support  systems:  the  Requirements 
Generation  System;  the  Acquisition  Management  System;  and  the 
Planning,  Programming,  and  Budgeting  System.  They  describe  the 
typical  program  as  it  passes  through  concept  exploration,  definition 
and  approval,  and  the  various  phases  and  milestones  associated  with 
system  development  and  operation. 

The  DOD  documentation  is  supplemented  by  Air  Force,  AFMC,  and 
ESC  regulations,  as  well  as  other  related  documentation.  The  ESC  regula¬ 
tions  emphasize  the  procedures  to  be  used  on  Air  Force  programs  to 
implement  the  government-wide  and  DOD  acquisition  policies  and 
regulations.  The  combined  DOD  and  Air  Force  acquisition  system  docu¬ 
mentation  is  extensive,  difficult  to  comprehend,  and  potentially  detri¬ 
mental  to  the  conduct  of  a  successful  program  if  improperly  applied.  On 
the  other  hand,  the  legal  requirements  must  be  satisfied.  Also,  the  disci¬ 
pline  intended  by  this  documentation  is  necessary  for  a  successful  pro¬ 
gram.  The  problem,  of  course,  is  to  meet  the  legal  requirements  and  at 
the  same  time  to  provide  the  needed  system  capability  in  a  timely,  cost- 
effective  way.  MITRE  must  understand  the  intent  of  this  documentation. 
MITRE  must  also  assist  the  program  director  in  judiciously  applying  it  to 
the  system  acquisition  program. 


The  acquisition  regulations  tend  to  fall  into  three  general  catego¬ 
ries.  The  first,  lower  level  group  tends  to  deal  with  the  physical 
world.  This  group  is  normally  referred  to  as  military  specifications 
and  standards,  or  MILSPECS  and  MIL- Standards  for  short.  For 
example,  radiation  hardening  requirements  may  reside  in  a  MILSPEC. 
The  documents  in  this  group  are  slow  to  change.  A  listing  of  53  of 
the  more  common  ones  is  contained  on  page  6-A-5  of  DOD  instruc¬ 
tion  5000.2.  A  second  category  of  acquisition  regulations  governs  the 
interactions  between  the  SPO  and  the  contractors.  Most  of  these 
documents  tend  to  change  only  when  affected  by  the  third  group.  The 
third  category,  however,  tends  to  change  quite  often  as  a  result  of 
congressional  and  other  high-level  initiatives  regarding  acquisition 
strategies  and  approaches.  As  systems  engineer,  MITRE  must  concern 
itself  with  all  three  levels  of  documentation. 

Needless  to  say,  requirements  dictated  by  imposing  government 
regulations  such  as  MILSPECS  cost  money.  The  contractor’s  time 
and  resources  are  spent  trying  to  meet  the  requirements  of  the  speci¬ 
fications.  The  government  and  MITRE  must  review  the  contractor’s 
work.  To  avoid  expenditures  that  do  not  contribute  to  achieving  the 
required  capability,  one  must  be  careful  to  impose  specifications  only 
if  they  are  required.  Even  when  a  specification  is  imposed,  MITRE 
must  be  prepared  to  help  tailor  the  specification  to  the  system  at 
hand.  At  a  global  level,  it  makes  no  sense  to  apply  the  same  stan¬ 
dards  to  commercial  hardware  and  militarized  equipment.  Will 
the  system  be  maintained  by  contractors  or  by  government  person¬ 
nel,  and  how  does  that  affect  documentation  and  test  equipment 
requirements.^  Some  of  the  questions  may  be  much  more  difficult  to 
answer.  Should  the  contractor  be  required  to  provide  documentation 
that  is  suitable  for  the  government  to  use  at  a  later  time  to  competi¬ 
tively  procure  additional  systems?  This  reprocurement  data,  as  it  is 
referred  to,  is  very  expensive  and  often  not  used.  How  does  one 
decide  that  the  government  will  never  reprocure  an  item?  Perhaps 
even  more  difficult,  how  does  one  get  all  parties  on  the  government 
side  to  agree  to  the  answer? 


In  a  more  technical  vein,  how  much  software  testing  is  required  to 
ensure  proper  system  operation?  Should  a  management  information 
system  be  subject  to  the  same  testing  as  a  missile  warning  system?  How 
secure  does  the  system  have  to  be?  How  will  the  system  security  be 
demonstrated?  Answering  questions  such  as  these,  and  those  mentioned 
above,  requires  two  classes  of  expertise — one  that  has  an  in-depth  under¬ 
standing  of  the  government’s  regulations,  and  a  second  that  understands 
the  needs  that  the  government  is  attempting  to  satisfy  through  any  given 
system  acquisition  program.  MITRE  has  expertise  in  both  areas  and  a 
responsibility,  as  systems  engineer,  to  apply  that  expertise  in  advising  the 
program  director  on  the  application  of  government  regulations. 

The  Corporation  must  work  closely  with  ESC  staff  groups  respon¬ 
sible  for  areas  such  as  reliability  and  maintainability,  system  costing, 
and  procurement.  The  MITRE  project  personnel  who  are  counter¬ 
parts  of  these  groups  should  strive  for  consensus  on  what  should  be 
required  in  a  given  system  acquisition  so  that  the  program  director  is 
receiving  common  advice  from  both  MITRE  and  the  ESC  staff.  At  the 
same  time,  important  differences  should  be  identified  and  referred  to 
the  program  director  for  decision. 

The  second  category  of  regulations — that  governing  the  contractual 
interactions  between  the  government  and  industry — is  also  an  impor¬ 
tant  area  of  MITRE  effort.  The  Corporation  prepares  the  technical 
specification  that  the  contractor  must  satisfy,  and  participates  in  the 
preparation  of  the  SOW  and  the  CDRL.  The  SOW  may  have  require¬ 
ments  on  systems  engineering,  software  development,  and  various 
kinds  of  system  and  subsystem  testing.  The  CDRL  establishes  the 
data  that  the  contractor  will  provide  and  that  must  be  reviewed  and 
either  accepted  or  rejected  by  the  government.  All  of  these  areas 
impinge  on  the  achievement  of  the  needed  capability,  and  therefore 
are  of  concern  to  MITRE  as  systems  engineer.  Other  areas  in  this 
category  that  are  of  concern  include  packaging  and  marking,  inspec¬ 
tion  and  acceptance,  schedules,  government-furnished  support  re¬ 
sources,  security  classifications,  and  the  award  fee  plan  if  one  is  to 
be  used. 


The  third  general  category  of  government  regulations  concerns 
general  system  acquisition  management  strategies.  They  tend  to 
change  as  weaknesses  in  an  existing  approach  in  specific  cases  are 
recognized  and  alternatives  to  overcome  those  weaknesses  are  man¬ 
dated  by  high  levels  of  government.  Unfortunately,  there  is  no  univer¬ 
sally  applicable  “right  way”  to  manage  an  acquisition  program.  Each 
program  is  different;  each  approach  is  more  applicable  in  some  cir¬ 
cumstances,  less  so  in  others.  Building  a  prototype  is  sensible  when 
one  is  not  sure  of  a  technical  approach,  a  waste  of  time  and  money 
when  one  knows  what  is  to  be  done  and  how  to  do  it.  Buying  some¬ 
thing  that  is  substantially  unknown  through  a  fixed  price  procure¬ 
ment  is  almost  guaranteed  to  produce  something  different  from  what 
was  planned,  or  require  more  time  and  money.  Therefore,  “fly  before 
buy”  or  “everything  fixed  price”  are  unreasonable  if  they  are  man¬ 
dated  on  every  acquisition  program. 

Regulations  at  this  level  tend  to  be  cyclical  in  nature,  driven  first 
by  the  experience  of  the  government’s  most  recent  acquisition  pro¬ 
grams,  and  second  by  the  knowledge  and  opinions  of  people  who 
change  quite  frequently  at  this  level  of  government.  This  cyclical 
nature  is  well  illustrated  by  the  repeated  changes  in  the  government’s 
position  on  whether  systems  should  be  procured  on  a  fixed  price  or 
cost  basis.  When  one  approach  does  not  seem  to  work,  another  is 
required,  sometimes  returning  to  one  used  in  the  distant  past. 

MITRE  must  fully  understand  the  regulations  in  this  category  as 
they  exist  at  any  given  time.  The  Corporation  must  also  appreciate 
the  potential  pitfalls  that  are  built  into  any  particular  approach.  This 
is  anoti  area  where  the  Corporation’s  long  experience  in  systems 
engineering  on  large  ESC  systems  is  helpful.  Having  lived  through 
earlier  programs,  MITRE  has  a  greater  appreciation  for  the  limita¬ 
tions  of  a  given  acquisition  approach  than  some  others  may  have. 

The  Corporation  should  identify  the  problems  that  exist  in  applying  a 
given  acquisition  approach  to  a  new  program.  Even  if  the  approach  is 
still  required,  MITRE  should  help  the  program  director  initiate  addi¬ 
tional  activities  to  minimize  the  impact  of  those  problems.  Perhaps 
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the  contractor  may  be  asked  to  perform  additional  tasks  as  insurance 
against  a  potential  problem,  or  perhaps  MITRE,  the  government,  or 
another  contractor  may  do  so. 

MITRE’s  role  is  to  help  the  government  achieve  a  needed  capability 
on  a  timely  basis  and  at  a  reasonable  cost.  The  Corporation  needs  to 
recognize  at  any  given  time  what  specific  action  would  best  achieve  that 
end.  Understanding  what  needs  to  be  done,  MITRE  must  relate  that 
action  to  the  regulations  that  govern  the  prcKurement.  The  question  to  be 
answered  is,  “How  can  this  specific  action  be  taken  while  satisfying  the 
requirements  of  the  existing  regulations.’”  Interestingly,  one  finds  that  the 
existing  regulations  are  really  quite  permissive.  The  Corporation  needs  to 
work  closely  with  the  program  director  and  the  ESC  staff  responsible  for 
these  areas  to  achieve  agreement  both  on  what  is  necessary  and  how  it 
can  be  accomplished  within  the  framework  of  the  government’s  regula¬ 
tions.  The  approach  requires  identifying  what  must  be  done  first,  then 
figuring  out  how  it  can  be  accomplished.  It  is  too  easy  for  someone  to 
say,  “You  can’t  do  that  because  the  regulations  forbid  it.”  It  takes  con¬ 
siderably  more  insight  to  know  what  must  be  done,  and  how  it  can  be 
accomplished  while  meeting  the  letter  of  the  law.  MITRE  must  be  famil¬ 
iar  with  the  regulations  to  help  get  done  what  needs  to  be  accomplished 
to  achieve  the  needed  capability. 


CbtpttrFive 


Four  Horsemen  of  C’CM 


Speciol  System  Considerotions 

A  theme  of  this  book  is  that  the  MITRE  approach  to  systems 
engineering  of  C^I  systems  is  based  on  two  important  qualities  found 
in  its  professional  staff:  a  thorough,  in-depth  appreciation  for  and 
understanding  of  the  operational  mission  capabilities  in  which  the  C^I 
systems  play  an  important  part;  and  skill  in  all  the  relevant  technolo¬ 
gies.  However,  systems  engineering  at  MITRE  involves  a  series  of 
professional  disciplines  that  are  not  limited  to  operational  and  techni¬ 
cal  considerations.  Some  of  these  important  disciplines  are  discussed 
in  this  chapter. 

The  subjects  discussed  below  tend  lo  fall  into  two  general  catego¬ 
ries.  The  first  category  is  driven  by  factors  external  to  the  program. 
Some  of  them  are  important  aspects  of  the  environment  in  which  the 
system  is  being  developed.  They  are  first-order  concerns  in  both  the 
initial  system  design  and  in  subsequent  systems  engineering  activities. 
They  are  not  static  or  one-time  challenges.  For  example,  the  threat 
can  change  quickly  and  dramatically,  as  recent  world  events  have  so 
clearly  emphasized.  To  be  effective  in  these  areas,  MITRE  staff  must 
remain  current  on  the  environment  in  which  the  acquisition  is  taking 
place,  as  well  as  the  one  in  which  the  system  will  eventually  operate. 


The  other  class  of  subjects  discussed  in  this  chapter  arc  internal 
system  design  factors  that  cut  across  the  individual  functions  of  a  C^l 
system.  Experience  dictates  they  can  cause  serious  system  perfor¬ 
mance  problems  if  taken  for  granted  in  the  system  development 
process.  These  include  functions  such  as  system  performance  moni¬ 
toring  and  system  exercising.  Clearly,  the  operating  command  must 
be  able  to  determine  whether  the  system  is  performing  as  required 
and  must  be  able  to  train  personnel  in  its  operation.  System  consider¬ 
ations  such  as  these  apply  to  all  the  C^I  functions  ranging  from  sur¬ 
veillance  to  information  processing  and  communication  of  results  to 
other  portions  of  the  mission  capability. 

To  be  effective,  the  MITRE  systems  engineering  team  must  include 
staff  with  expertise  in  the  areas  described  below.  Their  work  must  be 
an  integral  part  of  the  Corporation’s  systems  engir;eering  activities. 

liit«rop«fdHlity 

As  discussed  earlier  in  this  book,  the  DOD  acquisition  process 
provides  individual  pieces  of  an  operational  mission  capability,  such 
as  air  defense.  These  pieces,  or  subsystems,  are  then  combined  by  the 
user  command  to  achieve  the  required  capability.  Each  of  these  sub¬ 
systems — a  radar,  communication  equipment,  command  center,  or 
weapon  platform — is  often  referred  to  as  a  system.  One  sees  and 
hears  references  to  the  F-15  weapons  system  or  the  Milstar  communi¬ 
cations  system.  As  we  have  seen,  some  years  ago  a  term  was  invented 
to  provide  a  label  for  the  collection  of  such  systems  that  constitute 
the  overall  mission  capability,  i.e.,  the  system-of-systems.  An  interest¬ 
ing  discussion  of  the  management  challenge  represented  by  “supersys¬ 
tems,”  as  they  are  sometimes  called,  may  be  found  in  a  MITRE 
document  written  by  Dr.  N.  Waks.^"*  This  reference  points  out  that 
the  term  used  to  refer  to  such  systems  has  been  modernized  to 
“macrosystems”  when  referring  to  programs  of  the  scope  of  the 
Strategic  Defense  Initiative.  In  this  book,  the  term  “capability”  is 

N.  Waks,  The  Management  of  Very  Large  Technical  Systems,  M89-77,  The 
MITRE  Corporation,  Bedford,  MA,  January  1990. 
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used  rather  than  “system-of-systems”  and  includes  the  amalgamation 
of  many  systems.  It  provides  the  operational  emphasis  so  important 
to  communicating  the  ultimate  aim  of  systems  engineering;  mission 
system  capability. 

The  need  for  the  many  subsystems  and  systems  that  constitute  the 
total  mission  capability  to  effectively  interoperate  with  one  another 
represents  another  major  systems  engineering  challenge.  Kach  of  the 
items  is  acquired  by  a  different  government  program  office;  each  may 
be  built  by  a  different  corporation;  the  schedules  for  the  pieces  are 
rime-disparate;  and  there  is  no  requirement  to  coordinate  changes  in 
one  piece  with  the  interfacing  pieces  before  making  the  changes.  Once 
acquired,  each  piece  evolves  on  its  own  schedule,  sometimes  without 
the  involvement  of  the  development  community.  Evolutionarv,  incre¬ 
mental  improvements  provided  by  the  development  community  need 
to  accommodate  such  changes  as  they  are  made  by  the  user  command 
or  by  the  AFMC.  There  is,  indeed,  a  major  management  and  techni¬ 
cal  challenge  to  be  overcome  in  achieving  the  required  system-of- 
systems  capability. 


MITRE  has  had,  and  continues  to  have,  systems  engineering  re¬ 
sponsibility  for  many  of  the  C^I  subsystems  that  partially  constitute 
operational  capabilities,  such  as  strategic  defense  or  tactical  air  opera¬ 
tions.  C^I  systems  are  the  glue  that  ties  the  pieces  of  the  capability 
together.  They  provide  the  facilities  for  user  command  management 
of  the  operational  capability.  Because  of  MITRE’s  systems  engineer¬ 
ing  responsibility  on  so  many  of  the  C^I  subsystems,  and  because  of 
the  Corporation’s  dedication  to  helping  the  government  achieve 
required  capabilities,  MITRE’s  systems  engineering  staff  gives  special 
emphasis  to  the  interoperability  among  not  only  the  C^I  subsystems, 
but  also  between  the  C^I  systems  and  other  portions  of  the  capability, 
such  as  the  weapons  systems.  Interoperability  is  an  important  consid¬ 
eration  in  all  of  MITRE’s  systems  engineering  work. 

It  should  be  noted  that  the  challenge  of  interoperability  becomes 
greater  and  greater  as  joint  operations  involving  all  the  U.S.  military 
services  evolve  in  scope.  New  technology  has  accelerated  the  pace  at 
which  operations  may  be  conducted,  and  has  permitted  significant 
increases  in  the  demand  for  precise  control.  Both  of  these  facts  com¬ 
plicate  the  interoperability  problem.  Again,  MITRE  is  uniquely  posi¬ 
tioned  to  help.  These  requirements  put  a  major  burden  on  the  C^I 
systems — MITRE’s  forte.  The  Corporation  has  the  background  that 
comes  from  working  on  systems  employed  by  all  the  U.S.  military 
services  and  on  many  of  those  used  by  allied  nations.  That  experi¬ 
ence,  combined  with  the  Corporation’s  dedication  to  helping  the 
government  achieve  required  mission  capabilities,  provides  a  unique 
opportunity  and  responsibility  for  it  to  help  with  the  interoperability 
among  the  systems  that  constitute  the  capability. 

Clearly,  two  systems  that  are  physically  interconnected  must  pro¬ 
vide  for  compatible  electrical  characteristics  on  each  end  of  the  com¬ 
munications  lines  that  connect  them.  Ensuring  that  would  seem  like  a 
fairly  straightforward  problem.  It  is  not.  One  must  remember  that 
each  of  the  systems  is  being  developed  independently,  and  that  each 
has  its  own  constraints  in  time  and  money.  Each  acquisition  agency 
has  its  own  approach  to  minimizing  the  impact  on  the  program. 


especially  if  the  agency  is  committed  to  an  approach  and  the  develop¬ 
ers  of  a  new  system  wish  to  accomplish  the  interface  in  a  different 
wa/.  Even  the  relatively  simple  interface  problem  of  electrical  charac¬ 
teristics  gets  more  complicated  when  one  includes  international  sys¬ 
tems.  In  one  case,  MITRE  was  asked  to  help  achieve  an  interface 
between  two  national  systems  in  Europe.  Neither  country  wanted  to 
pay  for  it,  and  it  was  not  always  clear  that  both  really  wanted  the 
interface  to  work! 

But  achieving  an  effective  interface  is  more  than  compatible  electri¬ 
cal  characteristics.  Each  of  the  systems  has  to  speak  the  same  “lan¬ 
guage.”  If  two  systems  are  to  exchange  information  on  aircraft  being 
tracked,  each  must  be  able  to  accept  the  message  from  the  other, 
extract  the  information  on  track  position,  velocity,  identification,  and 
altitude,  and  store  the  result  in  its  own  track  storage  tables.  When 
one  system  identifies  an  aircraft  as  a  “faker,”  the  other  must  under¬ 
stand  the  meaning  of  that  classification.  One  must  be  concerned 
about  converting  from  the  coordinate  system  of  one  system  to  that  of 
the  other.  Life  gets  even  more  complicated  when  one  considers  opera¬ 
tions  in  which  the  control  facilities  of  one  service  or  one  nation  will 
be  employed  with  the  weapon  systems  another. 

To  the  extent  possible,  in  carrying  out  its  systems  engineering 
work,  MITRE  must  anticipate  interoperability  problems  and  incorpo¬ 
rate  the  resolution  into  the  system  technical  requirements.  The  Cor¬ 
poration  also  must  be  prepared  to  help  negotiate  changes  to  existing 
systems  necessary  to  accomplish  the  required  interfaces.  There  is 
another  alternative  for  creating  the  interface  that  seeks  to  avoid 
modifying  the  systems  that  must  interoperate.  This  involves  creation 
of  a  separate  interface  system  that  takes  information  from  one  sys¬ 
tem,  translates  it  into  information  that  can  be  understood  by  other 
systems,  and  then  forwards  it  to  those  systems.  The  translator  also 
processes  data  in  the  reverse  direction.  Such  an  approach  has  been 
used  when,  after  two  or  more  systems  have  been  built,  a  requirement 
is  established  for  them  to  interoperate.  Changing  the  existing  systems 
can  be  a  very  time-consuming  and  costly  process.  It  may  rake  a  very 


long  time  to  achieve  agreement  on  who  will  do  what  to  which  sys¬ 
tems,  to  decide  who  pays  for  what,  to  establish  contracts  to  do  so 
with  the  appropriate  industrial  corporations,  and  to  carry  them  out. 

The  compromise  alternative  is  another  system  to  achieve  the 
interface.  Again,  MITRE  has  considerable  direct  experience  with 
such  devices  for  each  of  the  U.S.  military  services,  and  between  U.S. 
systems  and  those  of  allied  nations.  For  ex.ample,  MITRE  built 
prototype  units  that  provided  an  early  interface  between  the  Air 
Force  407L  Tactical  Air  Control  System  and  the  NATO  NADGE  air 
defense  system.  These  prototypes  were  eventually  replaced  by  com¬ 
mercial  units.  The  interface  has  been  operationally  effective  and  the 
approach  adopted  was  more  practical  than  trying  to  change  both  of 
the  interfacing  systems  in  significant  ways.  However,  the  extra 
system  has  to  continue  to  be  supported  bv  ihe  user  commands  and 
upgraded  as  the  interfacing  systems  ate  modified  in  ways  that  affect 
the  interface. 

All  the  above  discussion  is  to  make  one  simple  but  crucial  point:  A 
very  important  part  of  MITRE’s  systems  engineering  activities  is 
attention  to  the  interoperability  among  the  various  systems  that 
constitute  the  mission  capability.  This  work  is  demanding,  tedious, 
and  never-ending.  It  requires  patience,  persistence,  dedication,  in- 
depch  knowledge  of  the  interfacing  systems,  and  most  of  all,  a  skill  at 
negotiating  with  the  many  players  involved. 
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Threat 

On  any  given  C^I  acquisition  program  for  which  MITRE  has 
system  engineering  responsibility,  its  staff  members  have  access  to 
government  intelligence  information  on  cu.r“nt  and  projected 
capabilities  of  potential  enemies  of  the  United  States.  This  access  is 
necessary  to  ensure  the  validity  of  the  system  requirements  re¬ 
flected  in  the  MiTRE-prepared  system  specification.  As  in  so  many 
other  areas  of  C^I  systems  engineering,  it  is  not  enough  that  some 
government  agency  reviews  a  draft  specification  and  requests 
changes  to  reflect  the  threat.  MITRE  staff  must  understand  the 


threat  and  must  remain  current  on  its  evolution.  There  are  simply 
too  many  occasions  as  the  CT  system  proceeds  day  by  day,  month 
by  month,  through  source  selection,  development,  testing,  and 
operation,  in  which  this  understanding  by  MITRE  can  help  ensure 
that  decisions  are  made  in  the  best  interest  of  achieving  the  capa¬ 
bility.  In  the  systems  engineering  role,  the  Corporation  participates 
in  every  important  program  action.  Those  agencies  that  are  devoted 
to  gathering  intelligence  information  and  assessing  potential  enemy 
capabilities  are  simply  not  involved  in  that  way.  The  potential 
threat  is  a  major  driver  in  determining  the  capabilities  required  in 
friendly  systems,  and  like  so  many  other  factors,  the  threat  evolves 
with  t  ine,  as  does  the  capability  itself.  It  is  therefore  crucial  that 
the  MITRE  staff  has  access  to  the  latest  intelligence  estimates  of 
existing  and  potential  enemy  capabilities. 

Even  in  these  days  of  rapid  prototyping  and  incremental  evolution¬ 
ary  development,  it  may  take  many  years  to  achieve  a  desired  level  of 
operational  mission  capability  for  C'l  system  components.  As  a 
result,  MITRE  must  pay  special  attention  to  potential  future  threats. 
But  that  is  an  art  itself,  and  people  can  become  specialists  in  this 
area.  MITRE  lias  been  fortunate  since  its  earliest  days  to  have  had 
projects  with  agencies  involved  in  the  analysis  of  enemy  capabilities. 
This  work  has  evolved  over  the  years  to  encompass  two  kinds  of 
activities.  One  that  developed  after  the  Corporation  existed  for  sev¬ 
eral  years  involves  helping  agencies  acquire  C’l  systems  in  a  manner 
similar  to  the  assistance  that  MITRE  provides  to  the  Air  Force  and 
other  DOD  services  and  agencies. 

However,  the  most  interesting  activity  in  this  section  has  existed 
almost  from  the  very  beginning  of  the  Corporation  and  has  been 
continuous  in  all  the  intervening  years.  For  example,  intelligence 
agencies  recognized  that  the  Corporation’s  expertise  in  air  defense 
was  applicable  to  their  task  of  analyzing  the  capabilities  of  potential 
enemy  air  defense  systems.  More  specifically,  under  East  Wing,  a 
joint  Air  Force-MITRE  project,  MITRE  staff  members  have  been 
analyzing  the  forces  and  the  systems  of  the  Soviet  Union  for  over  25 


East  Wing  Briefing  on  Soviet  Air  Defense 


years.  Special  emphasis  has  been  placed  on  Soviet  air  defense  systems. 
The  Corporation’s  work  in  analyzing  Soviet  and  other  potential 
enemy  systems  helps  to  define  the  capabilities  that  must  be  inherent 
in  U.S.  C^I  systems.  This  work  is  an  important  effort  in  ensuring  that 
such  capabilities  are  matched  to  the  real  threat  posed  by  enemy 
forces.  It  also  keeps  MITRE  current  on  the  state  of  the  art  in  C^I 
development,  and  therefore  better  able  to  provide  the  required  sys¬ 
tems  engineering  support.  In  addition,  the  results  of  the  MITRE 
analyses  are  valuable  to  the  Air  Force  in  non-C^I  areas  as  well.  For 
example,  the  capabilities  required  in  Air  Force  weapon  systems  are  in 
part  determined  by  the  expected  performance  of  the  C^I  systems  of 
potential  enemies. 

In  addition  to  work  for  the  Air  Force,  MITRE  has  also  assisted 
other  government  intelligence  agencies  in  acquiring  required  C^l 
systems  and  performing  technical  analyses  of  various  classes  of  in¬ 
telligence  data.  When  appropriate,  this  information  is  applied  to 
mitre’s  C*I  systems  engineering  projects. 

Obviously,  when  assessing  the  threat  and  applying  the  results  to 
the  design  of  C’l  systems,  one  must  avoid  underestimating  the  enemy 
capability.  On  the  other  band,  it  is  equally  important  that  MITRE 
staff  avoid  specifying  a  C^I  system  based  on  an  exaggerated  “ten-foot 
tall”  enemy.  In  assessing  potential  enemy  capabilities,  a  broad  view 
must  be  taken.  One  cannot  be  driven  solely  by  what  might  be  done 
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an  extreme,  that  logic  results  in 
a  paralyzed  acquisition  system. 


since  there  is  also  a  pragmatic  side  to  the  enemy’s  ability.  The  likeli¬ 
hood  the  enemy  will  take  some  action  or  other,  and  will  be  successful 
in  doing  so,  involves  the  overall  demands  he  must  satisfy,  his  techni¬ 
cal  capabilities,  manufacturing  skills,  etc.  The  enemy’s  estimate  of  the 
value  of  the  friendly  system  and  his  cost  to  do  something  to  counter 
it,  are  o’^her  important  factors.  In  the  end,  the  explicit  and  implicit 
threat  assumptions  that  go  into  the  design  of  a  CM  system  are  a 
complex  matter  of  economics  and  value  judgments  on  both  the 
friendly  and  enemy  side.s.  Decisions  by  one  side  or  the  other  can 
affect  a  C^I  system.  MITRE  must  remain  current  on  such  factors 
throughout  the  life  of  an  acquisition  program  and  work  with  the 
government  program  director  to  reflect  them  appropriately  into 
the  C^I  system. 

A  system  designed  to  meet  an  overestimated  threat  will  certainly  be 
unnecessarily  expensive  and  may  well  encounter  serious  development 
problems.  Neither  result  reflects  well  on  the  systems  engineer.  Mak¬ 
ing  decisions  on  the  level  of  threat  to  reflect  in  the  system  specifica¬ 
tion  is  a  difficult  task.  It  requires  close  coordination  with  the  user 
command,  development  agencies,  and  intelligence  community.  It  is 
fortunate  that  MITRE  has  professional  staff  skilled  in  that  process, 
with  many  years  of  experience  in  doing  it,  and  cognizant  of  the  latest 
information  through  the  Corporation’s  work  with  the  agencies  re¬ 
sponsible  for  assessing  enemy  capabilities.  This  combination  is  unique 
to  MITRE  and  essential  to  MITRE’s  effectiveness  in  the  systems 
engineering  role. 

There  is  another  potential  pitfall  to  avoid  in  assessing  enemy 
capabilities  as  an  input  to  the  design  of  a  friendly  C^l  system.  It  is 
perfectly  possible  to  conjure  up  an  enemy  capability  that — if  it 
existed — would  negate  the  capability  of  the  CM  system  under  consid¬ 
eration,  and  therefore  say  that  the  system  should  not  be  built.  Carried 
to  an  extreme,  that  logic  would  result  in  a  paralyzed  acquisition 
system  in  which  no  capabilities  were  acquired  because  an  offsetting 
threat  can  always  be  conceived.  Countermeasures  that  an  enemy  may 
take  are  discussed  as  a  systems  engineering  consideration  in  the  next 


section.  However,  it  is  important  to  note  here  that  there  is  never  a 
single  threat  to  which  a  C^l  system  must  respond.  A  system  such  as 
AW  ACS  must  be  capable  of  operating  in  a  wide  range  of  scenarios 
and  circumstances.  The  fact  that  one  may  be  able  to  define  threat 
conditions  that  might  reduce  the  capability  of  a  planned  C^l  system 
does  not  mean  that  the  system  should  not  be  built.  One  cannot  afford 
to  design  for  extreme  or  remote  circumstances.  Indeed,  the  system 
engineer  and  the  other  program  participants  need  to  consider  the 
range  of  circumstances  under  which  the  system  may  operate  and 
assess  the  contribution  of  the  system  to  the  operational  mission 

capabilities  under  those  circumstances.  This  may  indeed  lead  to  a  t.  r  ■  .  n 
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One  must  also  recognize  that  the  circumstances  outside  the  pro¬ 
gram  can  be  modified  to  make  the  system  viable  in  a  wider  variety  of  pqI  |)0  |)uj|f 

situations.  Again,  to  use  AWACS  as  an  example,  physical  survivabil¬ 
ity  of  the  platform  in  contested  air  space  is  of  major  concern.  The 
user  command,  however,  can  decide  to  allot  some  of  its  friendly 
fighter  aircraft  to  protecting  the  AWACS  from  attack  by  enemy 
aircraft,  thereby  extending  the  circumstances  in  which  AWACS  is  a 
survivable  C^I  system.  This  simple  example  again  emphasizes  the 
close  and  continuous  interaction  that  needs  to  take  place  among 
MITRE,  the  development  agency,  and  the  operational  user  command 
to  maximize  the  utility  of  U.S.  C^I  systems. 

As  systems  engineer,  MITRE  must  understand  the  evolving  threat 
in  detail  and  must  be  prepared  to  reflect  it  in  its  recommendations  as 
the  C^I  system  development  proceeds.  As  always,  the  question  that 
MITRE  must  continuously  answer  is,  “All  things  considered,  what 
action  should  be  taken  at  this  time  to  achieve  the  required  capability 
on  a  reasonable  schedule  and  at  a  reasonable  cost?” 


CountenMasures  aad  Co4Mter-(OMrtenM«s«Mres 
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This  section  expands  one  aspect  of  the  threat  discussed  above.  Any 
time  an  adversary  achieves  a  capability,  or  is  in  the  process  of  devel¬ 
oping  a  new  one,  friendly  forces  begin  to  think  about  and  take  action 
to  counter  that  capability,  or  to  somehow  overcome  whatever  advan¬ 
tage  the  new  system  gives  the  enemy.  The  enemy  acts  in  a  similar  way 
in  response  to  systems  built  by  the  United  States.  An  important  aspect 
of  C’l  systems  engineering  is  consideration  of  how  the  enemy  might 
choose  to  counteract  the  system.  Are  they  likely  to  try  to  physically 
destroy  the  system  or  render  it  inoperable  by  jamming.^  Perhaps  they 
will  choose  to  try  to  overwhelm  it  by  introducing  false  targets  or 
decoys.  They  might  even  try  to  exploit  the  system  to  their  own 
advantage  by  intercepting  information  as  it  is  processed  and  commu¬ 
nicated  by  the  C^I  system.  These  potential  tactics — destruction,  jam¬ 
ming,  spoofing,  and  exploitation — have  come  to  be  known  as  the 
“Four  Horsemen”  of  countermeasures  (C^CM).  In  contemplating 
potential  countermeasures,  MITRE  staff  must  remember  both  the 
highly  sophisticated  threats  and  the  very  mundane.  Creating  smoke 
may  be  as  effective  as  building  decoys,  and  it  is  much  easier  to  do. 

In  escalating  this  process,  the  friendly  forces  may  then  begin  to 
define  actions  they  might  take  to  counter  the  enemy  countermeasures, 
hence  the  term  “counter-countermeasures.”  Since  such  considerations 
are  central  to  the  operational  viability  of  a  C^I  system,  they  are 
important  factors  in  MITRE’s  C^I  work.  Again,  the  Corporation  enjoys 
the  distinct  advantage  of  having  a  group  of  experts  in  these  fields  who 
have  been  working  with  intelligence  agencies  in  assessing  potential 
enemy  capabilities  and  with  U.S.  development  agencies  in  defining  and 
acquiring  capabilities  to  overcome  potential  enemy  initiatives  of  this 
sort.  MITRE’s  work  has  made  specific  contributions  to  systems  de¬ 
signed  to  defeat  potential  enemy  C^I  systems.  C^CM  tactics  were 
employed  effectively  by  U.S.  forces  in  Operation  Desert  Storm. 

This  aspect  of  systems  engineering  involves  a  serious  contest  of 
point  and  counterpoint  and  counter-counterpoint.  Again,  carried  to 
an  extreme,  such  a  process  could  lead  to  a  failure  to  take  prudent 


acquisition  actions  for  fear  that  the  enemy  will  do  something  to 
counter  the  system  being  considered.  The  considerations  of  what  to 
do  are  in  part  the  same  as  those  discussed  above.  One  can  change  the 
design  to  actively  counter  potential  enemy  countermeasures.  For 
example,  communications  can  be  encrypted  to  avoid  enemy  intercept 
and  exploitation.  Or  one  could  employ  another  system  to  negate  the 
enemy  countermeasures  by  physically  attacking  their  facilities,  or  by 
jamming  them  or  spoofing  them.  In  all  of  this,  MITRE’s  systems 
engineering  staff  must  be  especially  wary  of  equivalent  actions  by  the 
potential  enemy,  especially  those  designed  to  negate  or  spoof  the 
friendly  C^I  systems. 

Survivobility 

Another  important  aspect  of  the  threat  relates  to  the  physical 
survivability  of  the  C^I  system.  Electronic  survivability — jamming, 
exploitation,  and  spoofing — has  already  been  covered.  However, 
physical  and  functional  survivability  despite  enemy  attack  deserve 
some  further  discussion.  There  are  several  system  design  ap¬ 
proaches  that  may  be  considered.  An  obvious  one  is  redundancy — 
an  approach  in  which  the  potentially  vulnerable  portions  of  the  C^I 
system  are  replicated  so  that  if  one  is  destroyed,  another  may  take 
over  its  function.  Another  is  to  make  the  facilities  transportable; 
that  is,  they  can  be  moved  from  place  to  place  to  make  the  enemy’s 
targeting  problem  more  difficult.  Or  they  can  be  made  mobile.  A 
mobile  system  can  be  moved  and  can  operate  while  moving.  De¬ 
coys  can  be  deployed  to  increase  the  enemy’s  targeting  problem. 
Alternatively,  one  can  employ  a  variety  of  hardening  techniques, 
such  as  underground  bunkers.  Each  of  these  alternatives  has  its 
disadvantages;  all  of  them  cost  money  to  acquire,  as  well  as  to  own 
and  operate.  Redundant  facilities  require  more  operating  and  main¬ 
tenance  personnel,  as  well  as  more  support  facilities.  They  also 
req'Mre  complex  operating  arrangements  to  ensure  continuity  of 
operation  during  and  after  an  attack.  Transportable  systems  have 
little  or  no  capability  while  in  transit.  Packing  up,  moving,  and 
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setting  up  again  is  a  complicated  process  in  which  things  tend  to 
get  broken  or  lost.  Mobile  systems  are  expensive  to  own  and  oper¬ 
ate,  especially  if  they  achieve  increased  physical  survivability  by 
operating  from  airborne  platforms. 

As  in  most  aspects  of  systems  engineering,  there  is  no  “school 
solution,”  no  single  answer  that  fits  all  situations.  Each  case  must  be 
carefully  studied  on  its  own  merits.  How  critical  is  it  that  the  capabil¬ 
ity  be  continuously  available?  What  can  be  done  to  actively  protect 
the  C’l  system,  as  in  the  case  of  fighter  aircraft  protecting  AWACS? 
How  can  the  enemy  problem  be  made  more  difficult?  What  are  the 
risks  and  the  costs  associated  with  the  possible  alternatives? 

Again,  the  MITRE  staff  has  wide  experience  in  such  assessments. 

In  its  early  work  on  survivable  air  defense  systems,  the  Corporation 
was  system  engineer  on  programs  such  as  the  Super  Combat  Center, 
which  built  an  underground  facility  in  North  Bay,  Ontario,  Canada, 
and  in  the  425L  program,  which  built  the  underground  facilities  at 
NORAD.  More  recently,  the  Corporation  had  similar  responsibilities 
for  the  new  underground  operations  center  at  Strategic  Air  Command 
(SAC)  Headquarters.  MITRE  was  the  system  engineer  on  the  BUIC 
system  that  provided  air  defense  C^I  survivability  through  redun¬ 
dancy  of  the  control  facilities.  MITRE  and  ESC  have  recently  been  a 
part  of  programs,  such  as  one  for  SAC  in  which  transportable  control 
facilities  were  provided.  MITRE  has  had  systems  engineering  respon¬ 
sibility  for  airborne  C^I  systems  such  as  the  Advanced  Airborne 
Command  Post,  AWACS,  and  the  Joint  Strategic  Target  Attack  Radar 
System  (Joint  STARS).  And  for  more  than  25  years,  MITRE  has  had 
systems  engineering  responsibility  for  a  series  of  rapidly  deployable 
C^I  systems  for  use  by  the  U.S.  tactical  Air  Forces  anywhere  in  the 
world.  As  a  group,  the  MITRE  staff  has  had  firsthand  systems  engi¬ 
neering  responsibility  for  systems  that  employ  the  various  alternatives 
available  for  addressing  the  requirement  for  physical  survivability  of 
C’l  systems.  This  experience  is  invaluable  in  making  meaningful 
recommendations  concerning  the  approach  to  physical  survivability  in 
the  design  of  new  C’l  systems. 


The  anticipated  physical  threat  can  also  include  damage  due  to 
radiation  effects  of  nuclear  explosions.  This  area  has  been  of  major 
concern  to  the  MITRE  systems  engineering  staff  on  projects  such  as 
the  Advanced  Airborne  Command  Post  and  the  Milstar  satellite 
communications  system.  The  Corporation  has  laboratories  in  which 
relevant  measurements  can  be  made.  MITRE  staff  members  have  also 
participated  in  tests  at  facilities,  such  as  the  Air  Force  trestle  facility 
used  to  test  the  Advanced  Airborne  Command  Post  aircraft  and 
equipment  for  radiation  hardening. 

One  observation  that  should  be  made  in  considering  design  for 
physical  survivability  is  the  necessity  for  a  sensible  balance  between 
the  various  components  of  the  overall  capability.  An  air  defense 
system  might  consist  of  sensors,  communications,  command  and 
control  centers,  and  weapon  systems.  In  considering  the  survivability 
of  the  air  defense  system,  one  needs  to  ensure  that  there  is  a  healthy 
balance  among  these  functional  areas.  A  system  in  which  the  commu¬ 
nications  all  pass  through  a  small  number  of  nodes  is  not  survivable 
even  if  there  are  many  redundant  sensors,  control  centers,  and  weap¬ 
ons  bases.  In  such  a  case,  the  enemy  has  an  obvious  weakness  to 
exploit  when  trying  to  degrade  the  system  capability.  On  the  other 
hand,  the  subsystems  of  a  total  system  capability  such  as  air  defense 
are  not  normally  acquired  concurrently.  It  may  well  be  prudent  to 
increase  the  number  of  control  centers  the  enemy  must  target  even 
when  the  communications  systems  are  vulnerable,  as  long  as  there  is 
a  plan  for  restoring  balance  among  the  major  subsystems  as  time  and 
money  permit.  MITRE’s  dedication  to  achieving  the  required  mission 
system  capability  demands  that  the  MITRE  staff  be  knowledgeable 
on  all  aspects  of  system  survivability  and  prepared  to  recommend 
actions  necessary  to  achieve  it. 

PerfornHHKe  Monitoriiig 

Many  of  the  existing  C^I  systems  are  extremely  critical  to  the 
nation’s  defense.  To  appreciate  that  fact,  one  need  only  consider  sys¬ 
tems  such  as  those  used  to  control  the  U.S.  nuclear  strike  capability. 


These  systems  are  very  extensive  and  very  complicated.  It  almost  goes 
without  saying,  then,  that  the  user  command  must  be  provided  with 
the  means  to  know  that  its  CM  systems  are  operating  properly.  Another 
key  ingredient  of  MlTRE’s  systems  engineering  work  is  to  ensure  that 
the  using  command  personnel  can  monitor  the  performance  of  their 
C^I  systems  with  high  confidence. 

One  approach  is  to  provide  a  series  of  tests  that  the  user  can  em¬ 
ploy  to  measure  whether  a  system  is  operating  properly.  This  can 
mean  expensive  test  tools  and  complicated  procedures.  It  requires 
that  the  test  be  properly  conducted  to  avoid  misleading  results.  It 
sometimes  requires  that  while  the  testing  is  being  accomplished,  the 
system  is  not  available  for  operational  use.  All  of  these  aspects  are 
undesirable.  But  sometimes  they  cannot  be  avoided,  especially  when 
one  is  concerned  about  overall  performance  of  the  mission  system 
capability.  For  example,  to  determine  whether  U.S.  missile  warning 
systems  are  operating  as  planned  requires  a  very  elaborate  test  pro¬ 
gram.  MITRE  has  helped  define,  conduct,  and  analyze  such  tests. 

This  aspect  of  performance  monitoring  is  related  to  system  exercising 
and  performance  evaluation,  as  discussed  in  the  next  section. 

At  a  lower  level  of  aggregation,  one  can  provide  an  individual  CM 
system  with  an  inherent  capability  to  monitor  its  own  performance 
and  to  communicate  the  results  to  user  personnel.  One  common 
approach  is  the  use  of  built-in  test  equipment,  or  BITE.  The  use  of 
BITE  is  another  important  systems  engineering  consideration.  What 
measurements  will  convey  an  accurate,  timely,  and  affordable  esti¬ 
mate  of  current  system  performance?  What  are  the  criteria  for  decid¬ 
ing  when  a  failure  has  occurred,  versus  a  degradation?  How  does  one 
avoid  a  high  incidence  of  false  alarms?  This  latter  problem  has  been 
especially  vexing  and  has  required  a  redesign  of  the  BITE  in  some 
systems  to  make  it  operationally  effective.  How  does  one  avoid  mak¬ 
ing  the  BITE  so  complicated  that  another  system  is  needed  to  distin¬ 
guish  operational  system  failures  from  BITE  failures?  To  what  level 
does  the  BITE  have  to  identify  the  source  of  the  failure?  These  are 
some  of  the  key  questions  that  the  system  engineer  must  help  answer. 


Another  possibility  is  to  use  one  system  to  help  monitor  the  perfor¬ 
mance  of  another.  For  example,  in  some  air  defense  systems,  the 
aircraft  tracking  function  in  the  control  center  processes  the  data 
received  from  several  radars.  In  particular,  the  tracking  function  often 
has  data  from  several  radars  on  a  given  aircraft.  One  might  design 
the  software  in  the  tracking  program  to  compare  the  quality  of  the 
data  received  from  the  several  radars  and  to  identify  performance 
problems  in  one  or  more  of  them.  If  the  system  communicates  those 
results  to  operating  personnel,  they  can  be  addressed  and  the  prob¬ 
lems  resolved.  The  use  of  one  system  to  monitor  the  performance  of 
another  is  an  important  approach  but  one  that  is  often  overlooked. 
Requirements  for  such  capabilities  are  rarely  found  in  the  program 
direction  because  the  systems  are  developed  independently.  MITRE 
should  keep  this  idea  in  mind  as  it  provides  systems  engineering 
support  to  C^I  acquisition  programs.  Again,  one  must  be  concerned 
about  avoiding  undue  complication  of  the  system  while  providing 
capabilities  that  provide  high  confidence,  low  false  alarm,  estimates 
of  system  performance. 

System  Exerdsing  and  PerfonmuKe  EvahratioH 

In  the  preceding  section,  there  is  a  brief  discussion  of  the  impor¬ 
tance  of  system  performance  monitoring  functions  as  considerations 
in  MITRE’s  C^I  systems  engineering  work.  Two  related  system  func¬ 
tions  are  the  ability  to  exercise  the  system  to  maintain  high  operator 
proficiency,  and  an  ability  to  evaluate  not  just  whether  the  system  is 
operating  as  planned,  but  also  how  well  it  is  performing  the  opera¬ 
tional  mission  for  which  it  was  designed. 

It  is  possible,  and  often  desirable,  to  build  into  a  system  a  capabil¬ 
ity  to  train  system  operators.  For  example,  the  SAGE  air  defense 
system  had  a  Training  and  Battle  Simulation  (TBS)  function  internal 
to  the  system.  One  feature  of  the  TBS  system  permitted  intercept 
directors  to  be  trained  without  having  to  fly  live  aircraft.  The  com¬ 
puter  provided  realistic  displays  that  simulated  what  an  intercept 
director  would  see  in  an  actual  engagement.  The  intercept  director 


gave  voice  radio  guidance  commands  as  might  be  done  in  a  live 
situation,  and  a  remotely  located  pilot  simulator  entered  that  infor¬ 
mation  into  the  computer  system.  In  response  to  those  inputs,  the 
computer  moved  the  interceptor  track  display  on  the  display  console, 
thereby  simulating  what  the  operator  would  see  in  the  live  case.  This 
provided  on-the-job  training  that  was  important  to  system  proficiency 
and  that  would  have  been  very  costly  to  achieve  using  live  aircraft. 
Use  of  live  aircraft  would  also  have  introduced  unacceptable  safety 
concerns  when  the  intercept  directors  were  in  training  and  not  yet 
fully  qualified. 

Some  systems  have  an  operational  mission  all  day,  every  day.  One 
example  is  the  U.S.  system  for  air  traffic  control.  Training  for  such  a 
system  is  important.  It  is  equally  important  and  much  more  difficult 
to  achieve  for  a  system  that  only  operates  under  certain  circum¬ 
stances.  For  example,  a  deployable  tactical  air  control  system  might 
only  be  used  operationally  in  a  period  of  tension  or  crisis  in  some 
remote  part  of  the  world.  Maintaining  proficiency  is  difficult  when 
day-to-day  operation  is  not  required. 

The  very  act  of  operating  a  system  helps  maintain  operator  profi¬ 
ciency.  It  also  helps  to  identify  changes  in  procedures  that  would 
improve  system  performance.  Substitutes  for  that  must  be  available 
when  there  is  no  requirement  for  continuous  operation.  There  must 
be  a  capability  to  simulate  the  conditions  under  which  the  system 
must  be  able  to  operate.  Providing  such  a  capability  can  be  a  formi¬ 
dable  system  engineering  challenge.  The  National  Aeronautics  and 
Space  Administration  (NASA)  efforts  to  prepare  astronauts  for  space 
flight  help  calibrate  how  challenging  this  requirement  can  be.  Mili¬ 
tary  C^I  systems  are  no  exception.  For  example,  how  does  one  pro¬ 
vide  for  realistic  training  and  exercise  of  the  U.S.  system  for  strategic 
warning,  or  for  the  deployable  tactical  air  control  system? 

One  way  the  operating  commands  approach  this  problem  is  by 
conducting  exercises  of  several  kinds  ranging  from  limited  tests  of 
functions  such  as  communications,  to  full-scale  field  tests  under  as 
realistic  conditions  as  can  be  established.  The  exercises  can  range 


from  the  very  simple — pick  up  the  phone  and  see  if  you  can  reach  the 
other  party — to  very  elaborate  system-wide  tests.  One  might  intro¬ 
duce  simulated  incoming  ballistic  missile  reports  into  the  front-end  of 
the  missile  warning  system  and  then  exercise  the  remainder  of  the 
system,  up  to  the  point  of  deciding  how  to  respond,  as  if  the  reports 
were  real.  Or  one  might  deploy  a  tactical  air  control  system  to  Eu¬ 
rope,  set  it  up  there,  and  operate  it  for  some  time  together  with  live 
aircraft  flights  in  practice  air  defense  and  air  offense  missions. 

Such  exercises  have  several  redeeming  features.  They  improve 
operator  training  and  proficiency.  They  work  out  the  kinks  in 
planned  operating  procedures.  More  than  that,  however,  they  help  to 
identify  any  shortcomings  in  system  performance  that  require  atten¬ 
tion.  For  that  reason,  they  are  especially  instructive  to  both  the  oper¬ 
ating  commands  and  the  development  community.  As  stressed  early 
in  this  book,  a  key  part  of  MITRE’s  system  engineering  portfolio  is 
an  in-depth  understanding  of  the  operational  mission  that  a  system  is 
designed  to  support.  Active  participation  in  system  exercising  is  an 
excellent  method  for  improving  the  Corporation’s  knowledge  and 
understanding  of  both  operational  missions  and  the  potential  re¬ 
quired  improvements  in  system  functionality.  Early  understanding 
also  speeds  the  process  of  satisfying  new  needs  when  they  are  estab¬ 
lished  by  the  using  commands. 

The  challenge  to  the  system  engineer  is  to  provide  support  for 
operator  training,  system  exercising,  and  system  evaluation.  Some 
portion  of  that  support  is  best  designed  into  the  system  itself. 
Scenario  generators  may  be  required.  Realistic  simulators  may  be 
necessary  when  live  inputs  are  not  practical.  Some  of  these  func¬ 
tions  may  require  special  test  equipment.  The  other  important 
aspect  of  this  problem  is  to  be  sure  that  means  are  provided  to 
monitor  performance.  Some  can  be  done  on  line  while  the  system  is 
in  operation.  The  system  itself  may  record  performance  data  for 
subsequent  analysis.  Clearly,  testing  is  only  useful  if  there  is  feed¬ 
back  in  training  or  in  performance  information  that  can  be  used  to 
evaluate  and  improve  system  operation. 


Again,  MITRE  must  include  considerations  of  system  training, 
exercise,  and  evaluation  as  an  integral  part  of  its  system  engineer¬ 
ing  activities  during  system  definition,  development,  and  opera¬ 
tional  phases.  Although  less  glamorous  than  some  of  the  other 
system  functions,  they  are  no  less  essential  to  effective  system 
operation. 


As  strange  as  it  seems,  in 
many  cases,  it  is  difficult  to 
achieve  agreement  on  when  a 
system  is  operationolly  useful 
and  when  it  is  not. 


Degraded  Operatioa 

At  any  given  time,  it  is  likely  that  some  parts  of  a  complicated  CM 
system  will  not  be  operational.  Failures  will  occur.  Portions  of  the 
system  may  be  taken  out  of  service  for  testing  or  for  modification. 
Enemy  action  may  destroy  or  damage  some  of  the  subsystems.  As 
strange  as  it  seems,  in  many  cases,  it  is  difficult  to  achieve  agreement 
on  when  a  system  is  operationally  useful  and  when  it  is  not.  If  a 
control  center  has  five  operating  positions  and  two  of  them  are  not 
working,  is  the  system  operational?  One  may  know  the  condition  of 
the  system,  and  different  people  may  have  varying  opinions  on  its 
operational  status.  Despite  that,  a  challenging  system  engineering 
activity  is  concerned  with  how  best  to  provide  for  useful  system 
operation  when  portions  of  the  system  are  not  available  for  whatever 
reason.  This  concern  with  degraded  operation  has  two  principal 
components.  First,  what  is  the  minimum  essential  level  of  operation 
that  the  user  command  wishes  to  have  even  under  conditions  of 
degraded  operation?  Second,  what  failure  modes  will  the  system 
have?  That  is,  how  will  the  system  continue  to  operate  as  certain 
systems  functions  fail  or  degrade? 

The  first  of  these  questions  addresses  the  user  requirements  for 
minimum  essential  system  operation.  If  system  operation  cannot 
take  place  without  communications  between  point  A  and  point  B, 
then  the  system  design  must  make  provision  for  that  link  to  be 
robust.  This  can  be  done  through  redundancy,  by  employing  mul¬ 
tiple  media,  and  perhaps  through  hardening  of  the  facilities. 
MITRE  must  study  such  requirements  very  carefully  and  discuss 
them  thoroughly  with  the  user  command  personnel.  They  must  be 
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reflected  in  the  system  performance  specification  and  their  imple¬ 
mentation  closely  monitored  throughout  the  acquisition  program. 

The  second  important  aspect  of  degraded  system  operation  in¬ 
volves  the  definition  and  accommodation  of  failure  modes  within 
the  system  operation.  If  there  is  a  central  computer  that  is  essential 
to  operation,  perhaps  the  system  should  include  a  collocated,  redun¬ 
dant  backup  computer.  It  may  be  that  the  system  itself  should 
automatically  monitor  the  performance  of  the  computer  currently 
controlling  the  operation  and  switch  to  the  backup  when  a  failure  is 
detected.  Perhaps  the  switchover  should  be  under  operator  control. 
Or,  rs  another  alternative,  the  situation  may  dictate  a  completely 
separate  facility  located  remotely  from  the  first  one.  Considerations 
such  as  these  are  both  very  complicated  and  very  important.  Achiev¬ 
ing  a  satisfactory  capability  in  this  area,  without  making  the  system 
unnecessarily  costly  or  complicated,  is  a  major  challenge.  Again,  to 
contribute  to  a  sensible  solution,  MITRE’s  systems  engineering  team 
must  understand  the  operational  needs  in  depth  and  appreciate  what 
can  realistically  be  done  in  distributing  this  function  among  person¬ 
nel,  hardware,  and  software. 

For  system  exercising,  provision  must  also  be  made  for  crew  train¬ 
ing  in  the  various  system  modes  of  degraded  operation,  and  for 
evaluating  system  performance  in  those  modes.  Degraded  modes  of 
operation  will  not  function  properly  in  operational  use  unless  they 
are  practiced  regularly. 

ReliaMHy,  MomtaiiidiilHy,  and  Availability 

Reliability  and  maintainability  are  two  very  important  factors 
in  all  of  MITRE’s  systems  engineering  work.  The  importance  of 
these  factors  is  widely  recognized  throughout  the  acquisition  com¬ 
munity,  and  many  special  efforts  are  undertaken  to  provide  reliable 
systems  at  affordable  costs.  These  efforts  vary  from  those  intended 
to  provide  highly  reliable  components  to  the  sorts  of  questions  just 
discussed  in  the  previous  section.  One  of  the  MITRE  technical 
centers  is  dedicated  to  reliability  and  maintainability.  Personnel 


There  can  be  no  reliability 
unless  it  is  designed  into  the 
system  irom  'iie  beginning. 


Whof  reolly  counts  is  whether 
the  system  is  ovoiloble  for 
use  when  needed. 


trom  that  center  are  assigned  to  each  major  MITRE  systems  engi¬ 
neering  project. 

This  section  is  not  intended  to  provide  a  treatise  on  the  impor¬ 
tance  of  reliability  and  maintainability  or  the  techniques  for  achiev¬ 
ing  it.  Here,  only  a  few  general  observations  will  be  made.  First, 
there  can  be  no  reliability  unless  it  is  designed  into  the  system  from 
the  beginning.  Some  aspects  of  that  requirement  at  the  system  level 
were  discussed  in  the  previous  section.  Other  related  remarks  were 
made  in  the  section  on  performance  monitoring.  Obviously,  a  sys¬ 
tem  device  will  have  reliabi  y  problems  if  the  components  them¬ 
selves  are  not  reliable.  Less  obviously,  devices  will  be  unreliable  if 
they  are  crammed  into  too  little  space,  or  if  they  do  not  adequately 
dissipate  the  heat  generated  in  their  operation,  or  if  the  circuits  arc 
designed  so  that  they  operate  at  the  margins  of  cheir  capabilities. 
These  are  important  design  considerations  as  MITRE  staff  members 
write  system  specifications  and  evaluate  the  specifications,  devices, 
and  systems  provided  by  industry.  The  approach  in  which  all  such 
factors  are  considered  throughout  the  development  process,  rather 
than  just  afterthoughts,  is  now  referred  to  as  integrated  product 
development. 

The  second  general  observation  is  that  what  really  counts  is 
whether  the  system  is  available  for  use  when  needed.  In  some  cases, 
availability  is  driven  by  factors  outside  the  boundaries  of  reliability. 
For  example,  reliability  is  an  important  factor  in  the  availability  of 
an  attack  aircraft  to  make  a  bombing  run.  However,  the  availability 
of  the  aircraft  to  make  bombing  runs  over  some  sustained  period  is 
the  ultimate  measure  of  operational  utility.  In  that  context,  avail¬ 
ability  is  a  product  of  many  factors  in  addition  to  reliability  of  the 
aircraft  itself.  A  major  factor  is  the  length  of  time  required  between 
the  return  of  the  aircraft  from  a  prior  mission  until  it  is  ready  to  go 
on  the  next  mission — the  so-called  turnaround  time.  Required  main¬ 
tenance  is  a  factor,  as  are  fuel  and  weapon  availability.  The  avail¬ 
ability  of  pilots  to  make  repeated  aircraft  flights  is  another  key 
aspect.  Indeed,  these  latter  factors  may  drive  system  availability. 
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This  does  not  in  any  way  diminish  the  importance  of  reliability  and 
maintainability.  It  simply  means  that  there  are  other  important 
systems  engineering  considerations  as  well,  because  the  objective 
is  to  attain  effective  operational  capability,  not  just  adequate  reli¬ 
ability.  Some  C^I  analogs  to  the  aircraft  case  are  discussed  in  the 
next  section. 

Supportability 

The  supportability  of  a  C^I  system  is  another  important  consider¬ 
ation  to  MITRE  in  the  systems  engineering  role.  This  term  covers  a 
range  of  factors,  and  they  may  vary  with  the  particular  system.  To 
help  illustrate  them,  consider  an  Air  Force  C^I  system  that  is  to  sup¬ 
port  tactical  air  operations  anywhere  in  the  world,  with  or  without 
the  systems  of  other  U.S.  services  and  with  or  without  the  systems  of 
friendly  allies.  This  means  many  different  interfaces  between  systems. 
It  also  requires  the  capability  to  operate  under  many  different  tem¬ 
perature  and  terrain  regimes.  But  more  to  the  point  of  this  section,  it 
requires  the  ability  to  support  the  system  under  a  wide  range  of 
circumstances. 

What  is  required  to  support  the  operating  and  maintenance  per¬ 
sonnel  with  housing,  food,  water,  medical  treatment,  and  a  long  list 
of  other  such  items?  How  will  the  security  of  the  system  and  its 
personnel — physical  and  other — be  provided?  What  about  the  care 
and  feeding  of  the  system  itself,  or  spare  parts  and  repair  facilities? 

Is  the  necessary  electrical  power  to  be  supplied  from  the  local 
economy,  or  does  it  have  to  be  generated  by  power  units  that  are 
part  of  the  system?  Where  will  the  fuel  come  from  to  feed  the  gen¬ 
erators  and  other  equipment?  What  can  be  done  to  reduce  the  signa¬ 
tures  for  enemy  targeting  that  are  represented  by  the  heat  given  off 
by  the  equipment  or  the  noise  of  the  generators  or  the  air  condition¬ 
ers?  How  are  facilities  to  be  heated  or  cooled?  Hov  will  resupply  be 
managed?  The  length  of  the  resupply  line  may  be  critical.  Will 
spares  and  other  items  be  prepositioned?  These  and  many  other 


questions  like  them  have  to  be  of  concern  to  MITRE  in  its  systems 
engineering  role  on  deployable  systems.  But  there  are  others. 

A  deployable  system  requires  transport  to  move  it  from  one  place 
to  another.  Within  an  operational  theater,  tactical  systems  move 
frequently.  There  must  be  a  capability  to  tear  the  system  down, 
package  it  for  transport,  move  it,  unpack  it,  and  set  it  up  for  opera¬ 
tion.  This  must  be  done  rapidly  and  with  as  little  requirement  as 
possible  for  additional  hardware  and  personnel  beyond  that  required 
for  actual  operation.  One  must  also  provide  for  protecting  the  system 
and  its  personnel  during  transit.  Supporting  such  an  operation  may 
require  previously  prepared  sites  with  accommodations  for  electrical 
power  and  fuel. 

The  movement  of  a  tactical  system  from  its  base  in  the  United 
States  to  a  location  in  a  foreign  country,  or  from  one  foreign  country 
to  another,  requires  many  of  the  same  considerations  surrounding  in¬ 
theater  movement.  But  the  demands  on  the  transportation  function 
are  substantially  increased.  The  equipment  and  its  support  are  very 
extensive,  bulky,  and  heavy.  Some  things  will  fit  in  some  transport 
aircraft,  others  will  not.  In  the  earliest  design  work,  MITRE  must  be 
concerned  with  these  sorts  of  problems  so  that  when  the  rime  comes, 
the  available  transport  will  be  able  to  handle  the  system  require¬ 
ments.  The  concerns  range  from  ensuring  that  the  size  of  the  system 
boxes  does  not  exceed  that  of  the  available  loading  doors,  to  the 
fraction  of  the  total  available  transport  that  will  be  consumed  by 
moving  the  system  and  its  personnel  to  a  foreign  country. 

The  examples  in  this  section  are  intended  to  make  clear  that  in  the 
systems  engineering  role,  the  MITRE  technical  staff  is  involved  with 
many  aspects  of  the  overall  system,  and  that  some  of  them  are  far 
removed  from  classical  hardware  and  software  engineering  problems. 
They  are,  nonetheless,  system  problems  of  direct  concern  to  MITRE. 
With  a  goal  of  helping  the  government  to  achieve  the  required  capa¬ 
bility,  anything  that  may  impinge  on  that  is  deserving  of  the 
Corporation’s  close  attention. 


SystMi  CMtiag 

As  in  the  case  of  the  reliability  discussion  above,  this  section  does 
not  attempt  to  discuss  system  cost  matters  in  any  detail.  MITRE  has 
a  technical  center  that  addresses  issues  of  costs  and  other  related 
matters  as  it  applies  to  the  C^I  systems  for  which  MITRE  has  systems 
engineering  responsibility. 

Almost  since  the  very  beginning  of  MITRE,  there  has  been  a 
group  of  professional  staff  dedicated  to  supporting  the  systems 
engineering  projects  in  matters  such  as  costing  and  scheduling, 
management  support  systems,  procurement-related  matters,  and 
acquisition  regulations.  Today,  personnel  from  the  Cost  Center 
participate  in  all  the  major  C^I  system  projects  at  MITRE.  Their 
participation  starts  very  early  in  the  process  of  establishing  a  new 
acquisition  program.  A  portion  of  the  information  provided  by 
MITRE  to  the  government  on  what  may  be  done  to  satisfy  a  new 
requirement  includes  MITRE  estimates  on  the  associated  costs  of 
such  a  capability.  These  estimates  are  based  on  what  similar  devices 
have  cost  in  the  past,  what  they  are  apt  to  cost  when  they  will  be 
purchased,  and  what  the  total  system  cost  is  likely  to  be  when  all 
the  relevant  factors  are  taken  into  account.  This  sort  of  support 
continues  throughout  the  life  of  the  program  as  contractors  are 
selected  and  as  requirements  for  changes  and  additions  are  defined. 
It  should  also  be  noted  that  the  expertise  of  the  MITRE  Cost  Center 
covers  software,  as  well  as  hardware.  The  Cost  Center  staff  works 
closely  with  their  government  counterparts  and  with  MITRE  experts 
in  the  various  technical  disciplines. 

A  few  general  points  are  made  here.  First,  in  considering  a  new 
system,  the  cost  of  acquisition  is  only  one  factor.  There  is  also  a  cost 
of  ownership.  What  will  it  cost  year  after  year  to  ov/n  and  operate 
the  system?  That  cost  is  in  personnel  as  well  as  dollars.  How  many 
people  with  what  types  of  skills  will  be  required  to  operate  and 
maintain  the  system?  There  are  even  more  global  questions  in  this 
area.  If  the  system  requires  personnel  with  skills  in  the  computer 
field,  will  the  Air  Force  personnel  system  be  able  to  supply  them 
when  the  demands  for  similar  skills  in  all  the  other  Air  Force  systems 


The  combined  MITRE  team  can 
provide  the  best  possible  cost 
estimotes,  ones  certainly  better 
thon  those  thot  could  be 
provided  by  any  single  group  of 
cost,  technology,  or  functional 
experts  operoting  alone. 


are  taken  into  account?  How  will  civilian  demands  affect  Air  Force 
availability  of  the  necessary  skilled  personnel? 

Another  general  observation  concerns  the  need  to  be  able  to  esti¬ 
mate  what  a  new  device  will  cost.  MITRE  cost  experts  maintain  a 
large  database  on  what  different  devices  with  a  given  functionality 
have  cost  in  the  past.  For  example,  they  have  good  data  on  the  cost 
of  many  different  U.S.  aircraft  voice  radios.  They  are  able  to  provide 
the  necessary  inflation  estimates,  and  they  know  the  impact  of  docu¬ 
mentation  and  spares  costs  as  a  function  of  acquisition  cost  of  the 
radios.  However,  the  real  challenge  comes  in  providing  good  cost 
estimates  for  radios  using  new  technology  and  having  new  functional¬ 
ity.  Those  estimates  are  worked  out  by  a  combination  of  MITRE  staff 
that  includes  people  conversant  with  the  new  technology,  the  Cost 
Center,  and  those  familiar  with  the  desired  operational  functionality. 
Again,  the  capabilities  of  the  combined  MITRE  team  are  effective  in 
providing  the  best  possible  cost  estimates,  ones  that  are  certainly 
better  than  any  that  could  be  provided  by  any  single  group  of  cost, 
technology,  or  functional  experts  operating  alone. 

There  are  other  essential  areas  in  which  the  MITRE  Cost  Center 
has  achieved  expertise.  Installing  C^I  system  capabilities  in  a  modern 
fighter  aircraft  is  a  very  challenging  and  costly  process.  Because  of  the 
potential  for  interference  with  other  aircraft  systems,  Kpcause  of 
the  limited  space,  electrical  power,  and  cooling  capacity  available  on 
these  aircraft,  such  installations  are  difficult.  That  is  true  even  when 
the  installation  is  part  of  the  original  fabrication  of  the  aircraft.  It  is 
much  more  difficult  when  one  tries  to  retrofit  something  such  as  a 
radio  and  its  associated  antenna  into  an  existing  aircraft.  The  space, 
cooling,  and  power  limitations  are  greater.  Quite  properly,  the  Air 
Eorce  is  reluctant  to  take  the  aircraft  out  of  service  while  the  installa¬ 
tion  is  being  made.  In  this  area,  the  MITRE  Cost  Center  has  accumu¬ 
lated  a  database  on  the  cost  of  installing  electronic  equipment  in 
various  types  of  aircraft.  They  also  have  data  on  power,  cooling,  and 
space  availability.  That  information  has  been  important  to  MITRE’s 
work  on  systems  such  as  Have  Quick,  JTIDs,  and  Joint  STARS. 


A  third  area  of  expertise  for  the  MITRE  Cost  Center  is  logistics 
support.  Again,  data  is  available  on  broad  questions,  such  as  how 
many  systems  are  required  to  maintain  a  24  hour-per-day  orbit  of 
airborne  surveillance.  At  the  other  end  of  the  spectrum,  information 
is  available  on  the  variance  in  the  cost  of  electronic  components  as  a 
function  of  the  reliability  required.  In  between,  they  have  data  to  help 
estimate  what  spares  will  be  required,  and  what  maintenance  and 
repair  facilities  will  be  needed  as  a  function  of  the  complexity  of  the 
system  and  its  availability  requirements.  Likely  documentation  costs 
and  training  costs  are  two  other  aspects  of  the  total  system  cost  with 
which  MITRE  staff  members  are  familiar. 

System  Security 

This  section  does  not  attempt  to  discuss  system  security  in  any 
detail.  MITRE  has  a  technical  center  that  addresses  issues  of  interest 
in  this  area  as  it  applies  to  the  C^I  systems  for  which  MITRE  has 
system  engineering  responsibility.  Some  relevant  references  are  pro¬ 
vided  in  the  Appendix.  As  described  there,  MITRE  enjoys  a  position 
of  national  leadership  in  the  computer  security  area.  The  Corporation 
has  done  much  of  the  advanced  work  on  compartmented  mode  work¬ 
stations,  which  can  simultaneously  process  information  with  different 
levels  of  classification.  MITRE  has  also  been  involved  in  the  applica¬ 
tion  of  crypto-security  systems  to  a  variety  of  communications  facili¬ 
ties.  As  mentioned  earlier  in  this  chapter,  the  Corporation  is  active  in 
related  activities,  such  as  countermeasures  to  enemy  attempts  to 
exploit  or  jam  U.S.  C^I  systems. 

The  purpose  of  this  section  is  to  remind  the  reader  that  system 
security  is  growing  dramatically  in  importance  and  scope  as  depen¬ 
dence  on  C^I  systems  escalates,  the  technology  used  in  C^I  systems 
evolves,  and  the  enemy  capability  to  exploit  such  systems  becomes 
more  sophisticated.  Breaking  the  security  of  German  communications 
was  an  important  contribution  to  the  Allied  war  effort  in  World  War 
II.  Communications  today  are  much  more  extensive,  and  have  longer 
range  and  considerably  greater  capacity  than  they  did  then.  Modern 


military  operations  continue  to  be  very  dependent  on  them.  Beyond 
that,  the  widespread  use  of  computers  creates  much  more  planning 
and  status  reporting  capability,  all  of  which  is  of  limited  use  unless  it 
is  communicated. 

Some  modern  communications  architectures  raise  special  problems. 
Point-to-point  communications  require  protecting  the  links.  Today’s 
local  area  or  long-haul  networks  have  the  equivalent  of  many  point- 
to-point  links  that  are  all  potentially  accessible  from  any  node  on  the 
network.  This  provides  an  opportunity  for  someone  to  try  to  access 
information  that  was  not  intended  for  that  person. 

Again,  the  MITRE  staff  expert  in  the  various  subjects  mentioned 
above  are  essential  to  the  Corporation’s  full-function  systems  engi¬ 
neering  team.  Systems  engineering  involves  much  more  than  the  latest 
technologies  and  system  design.  MITRE’s  unique  ability  to  help  in  the 
acquisition  of  C^I  systems  derives  in  large  part  from  a  staff  expert  in 
all  relevant  areas. 

The  material  covered  in  this  chapter  touches  only  briefly  on  a 
series  of  systems  engineering  concerns  that  do  not  fall  neatly  into 
some  of  the  more  conventional  technical  and  operational  disciplines. 
No  attempt  is  made  to  discuss  them  in  great  detail.  However,  there  is 
considerable  MITRE  documentation  related  to  several  of  these  sub¬ 
jects,  should  one  wish  to  learn  more  about  how  MITRE  approaches 
them.  The  Appendix  provides  some  references  to  that  documentation. 


ChapUrSix 
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j  Sufflfflory 

For  over  35  years.  The  MITRE  Corporation  has  had  a  close, 
long-term — indeed,  a  special  relationship  with  the  Air  Force.  The 
Corporation  was  formed  in  1958  at  the  request  of  the  Air  Force. 
This  request  recognized  the  unique  knowledge  and  experience 
within  the  nucleus  of  systems  engineering  talent  that  transferred 
from  MIT’s  Lincoln  Laboratory  to  the  new  corporation.  The  cor¬ 
porate  form  and  rules  under  which  MITRE  operates  were  con¬ 
ceived  to  permit  it  to  perform  a  unique  function.  With  Air  Force 
support,  that  role  has  been  extended  to  similarly  serve  other  parts 
of  the  government. 

In  cooperation  with  the  government,  the  Corporation  has  config¬ 
ured  itself  to  operate  in  a  conflict-free  way.  As  an  FFRDC,  MITRE 
operates  under  a  set  of  rules  designed  to  facilitate  its  systems  engi¬ 
neering  responsibility.  It  is  independent,  not-for-profit,  does  not 
enter  bid  competitions  with  industry,  does  no  commercial  work, 
and  does  not  manufacture.  It  accepts  work  only  after  the  govern¬ 
ment  and  corporate  management  have  agreed  the  work  is  appropri¬ 
ate.  All  parts  of  MITRE’s  operation  are  open  to  government 
review.  Personnel  are  appropriately  constrained  with  regard  to 


personal  outside  activities.  These  factors  permit  the  most  conflict- 
free  operation  possible.  Rather  than  being  limitations,  they  help 
make  it  possible  for  MITRE  to  be  effective  in  the  systems  engineer¬ 
ing  role  between  government  and  industry. 

MlTRE’s  charter  to  work  “in  the  public  interest”  is  closely 
monitored  by  a  prestigious  Board  of  Trustees  that  examines  the 
appropriateness  of  each  new  class  of  proposed  work.  The  Board 
also  regularly  reviews  the  adequacy  of  the  Corporation’s  perfor¬ 
mance  in  each  major  work  area.  Active  corporate  management 
involvement  on  each  MITRE  project  is  a  key  ingredient  in  the 
success  of  its  work.  Management  interaction  with  their  peers  in 
government  and  industry  also  improves  MlTRE’s  ability  to  perform 
in  the  systems  engineering  role. 

Through  the  cooperative  efforts  of  government  and  industry, 
there  are  many  very  capable  military  systems  operational  today. 
Over  the  last  35  years,  MITRE  has  been  privileged  to  fulfill  the 
role  of  systems  engineer  on  more  than  100  different  C^I  systems  for 
the  Air  Force  and  other  DOD  sponsors.  A  long  history  of  success¬ 
ful  performance  provides  MITRE  with  detailed  knowledge  of  the 
associated  operational  capabilities  and  needs,  proficiency  in  sys¬ 
tems  engineering  for  such  systems,  and  a  C^I-related  corporate 
memory  unmatched  by  any  other  organization.  To  complement 
these  skills,  there  is  a  vigorous  program  to  maintain  a  professional 
technical  staff  who  are  current  in  all  of  the  technologies  applicable 
to  C^I  systems.  The  MITRE  Technology  Program,  professional 
development  training  through  the  MITRE  Institute,  very  selective 
hiring,  staff  interaction  with  peers  in  academia  and  industry,  and  c. 
dedication  to  providing  the  tools  necessary  for  the  technical  work 
required  on  system  projects  are  all  facets  of  MlTRE’s  commitment 
to  maintain  a  high  quality  technical  staff  who  are  skilled  in  the 
latest  technologies  and  have  appropriate  industrial  experience. 

Joint  ESC/MITRE  application  of  the  criteria  found  in  ESCR  80-1 
to  each  new  project  ensures  a  challenging  work  program.  The  high 
quality  of  the  work  program  also  helps  to  attract  and  maintain  a 


skilled  professional  staff.  Analogously,  a  high  quality  staff  attracts 
challenging  work  programs. 

The  combination  of  the  corporate  structure,  a  highly  trained 
professional  staff,  in-depth  understanding  of  the  military  mission 
areas,  long  and  successful  experience  in  systems  engineering  of  C^l 
programs,  along  with  unparalleled  corporate  memory  in  these 
latter  two  areas,  uniquely  qualify  MITRE  to  perform  in  the  systems 
engineering  role  for  the  Air  Force  and  other  DOD  clients.  That 
special  relationship  established  by  the  Air  Force  in  1958  has 
proven  to  be  most  efficacious  for  the  development  of  effective  C^I 
systems.  It  provides  an  opportunity  to  influence  major  programs 
significantly,  and  MITRE  continues  to  respond  well  to  the  respon¬ 
sibility  that  accompanies  that  opportunity. 

Despite  its  designation  as  an  FFRDC,  MITRE  has  no  guarantee 
of  work  from  any  government  organization.  There  is  no  line  item 
in  any  government  budget  labeled  “MITRE.”  As  an  FFRDC, 
MITRE  will  not  enter  bid  competitions  with  industry.  Its  viability 
as  a  systems  engineering  organization  is  based  on  the  quality  and 
cost  of  work  it  performs.  It  is  subject  to  all  the  market  forces  that 
govern  the  establishment  of  a  quality  work  force  at  competitive 
costs.  To  attract  a  quality  staff,  the  work  must  be  challenging  and 
rewarding,  the  facilities  matched  to  accomplishing  the  work,  and 
the  salaries  and  benefits  competitive  with  industry.  To  be  competi¬ 
tive  with  other  sources  of  systems  engineering,  the  work  must  be 
well  done  and  the  costs  as  low  or  lower  than  potential  alternatives 
for  equivalent  work. 

Although  this  book  emphasizes  MITRE’s  systems  engineering 
activities,  it  also  fully  recognizes  the  crucial  roles  of  government 
and  industry  in  the  successful  acquisition  and  operation  of  C^I 
systems.  Government  personnel  establish  the  needs,  provide  the 
funding,  manage  the  acquisition  programs,  and  employ  the  result¬ 
ing  systems  to  achieve  the  required  operational  capabilities.  MITRE 
cannot  accomplish  these  functions.  Although  expected  to  help  in 
any  way  possible  and  to  take  the  initiative  in  all  technical  matters, 


the  Corporation  must  avoid  even  the  appearance  of  usurping  the 
responsibilities  of  a  program  director.  MITRE  strives  to  be  a  full 
partner  as  part  of  the  program  office  in  carrying  out  an  acquisition 
program. 

Similarly,  profit-making  industry — not  MITRE — provides  the 
actual  hardware  and  software  integral  to  these  systems.  MITRE  has 
neither  the  charter  nor  the  resources  to  provide  these  things.  With¬ 
out  industry,  the  combination  of  government  and  MITRE  cannot 
provide  the  needed  capabilities.  The  Corporation’s  role  is  to  work 
between  government  and  industry  as  a  catalyst,  an  honest  broker, 
in  helping  to  achieve  the  capability.  When  performing  effectively  in 
that  role,  its  efforts  are  as  helpful  to  industry  as  they  are  to  the 
government.  MITRE  is  not  in  the  business  of  harassing  industry, 
but  rather  strives  to  help  government  and  industry  achieve  the 
needed  capabilities. 

The  environment  within  which  military  and  other  government  C^I 
systems  are  defined,  acquired,  and  operated  is  challenging.  It  is 
competitive — each  military  program  vies  with  others  for  resources, 
-^■'d  the  military  competes  with  social  services  and  other  important 
government  needs.  It  is  large,  complex,  and  extremely  dynamic. 

Many  people  in  important  positions  are  involved,  each  with  strong 
opinions  and  in  a  position  to  affect  significantly  the  course  of  the 
program.  The  people  in  these  positions  tend  to  change  quite  often, 
and  the  new  people  often  have  different  opinions  than  their  predeces¬ 
sors.  Technology  advances  while  enemy  capabilities  and  threats  vary 
over  time.  New  and  modified  operational  requirements  may  arise.  All 
these  factors  contribute  to  an  extremely  complicated  management 
challenge  for  the  government  acquisition  program  director  and  hence 
for  MITRE  in  providing  advice  to  the  director  and  the  integrated 
product  team.  An  additional  complication  of  the  environment  is  that 
every  new  system  must  interoperate  with  other  existing  and  planned 
systems,  each  of  them  equally  subject  to  the  environment,  if  an  effec¬ 
tive  military  mission  capability  is  to  result.  MITRE’s  experience  on 
so  many  of  these  systems  is  especially  valuable  in  this  regard. 


To  be  effective,  the  MITRE  systems  engineering  team  must  fully 
understand  the  environment — the  existing  systems,  all  of  the 
planned  systems  with  which  the  one  they  are  working  on  must 
interface,  the  status  of  each  of  these  systems,  all  of  the  players  that 
can  affect  the  program  and  direction  they  may  have  given,  re¬ 
sources  available  to  the  program  director,  the  status  of  industry 
and  government  efforts  on  the  program,  developing  technology  that 
may  be  applicable,  and  the  evolution  of  the  enemy  threat  to  the 
planned  capability.  Thorough  understanding  of  that  evolution, 
gained  through  many  years  experience  in  analyzing  enemy  systems 
for  ESC  and  other  government  agencies,  is  another  unique  MITRE 
advantage  in  systems  engineering  of  CM  systems. 

The  day-to-day  MITRE  systems  engineering  job  is  to  be  conver¬ 
sant  with  the  military  capability  that  the  acquisition  program  is  to 
help  provide,  continuously  conscious  of  the  environment  within 
which  the  program  is  taking  place,  and  constantly  assessing  what 
needs  to  be  done  to  maximize  the  probability  of  achieving  the 
required  capability  on  a  reasonable  schedule  and  at  a  reasonable 
cost.  The  MITRE  team  must  take  the  initiative  to  advise  the  inte¬ 
grated  product  team  on  any  actions  required  by  the  Corporation  or 
by  another  team  member  to  achieve  that  goal.  The  advice  must  be 
forthright,  implementable,  and  above  all,  correct.  MITRE  must  be 
careful  to  weed  out  the  trivia  and  to  emphasize  actions  important 
to  the  success  of  the  program. 

As  required  by  ESCR  80-1,  on  each  project,  MITRE  must  deliver 
the  systems  engineering  products  called  for  in  the  contract  covering 
the  work.  However,  as  programs  evolve  and  the  program  director 
and  the  MITRE  project  leader  agree  a  change  in  the  products  is 
necessary,  the  Corporation  is  able  to  quickly  redirect  its  staff  to 
match  the  program  dynamics.  This  is  another  unique  MITRE  ad¬ 
vantage  in  the  systems  engineering  role. 

mitre’s  staff  takes  a  broad  systems  view,  but  each  is  expert  in 
one  or  more  of  the  technologies,  operational  mission  areas,  or 
management  tools  relevant  to  CM  system  engineering.  The  technical 


staff  gain  this  knowledge  and  experience  in  various  ways.  Techni¬ 
cal  training;  experience  on  systems  engineering  projects;  interac¬ 
tions  with  experts  both  in  MITRE  and  elsewhere,  including  user 
command  personnel;  working  at  operating  locations;  participating 
in  test  activities  at  operational  sites  and  in  various  other  test  loca¬ 
tions,  including  contractor  plants;  all  help  to  increase  staff  capa¬ 
bilities.  MITRE’s  staff,  like  all  professionals,  sometimes  needs 
special  facilities  to  derive  the  answers  to  important  technical  ques¬ 
tions.  These  might  include  simulation  programs,  test  facilities,  or 
computer  analysis  programs.  The  technical  questions  that  must  be 
answered  should  be  identified  first,  and  only  then  the  facilities  that 
might  be  required  to  answer  them. 

MITRE’s  specific  systems  engineering  activities  for  each  phase  of 
an  acquisition  program  are  reviewed  in  Chapter  2.  They  include 
analysis  work  prior  to  the  initiation  of  a  new  program,  activities 
during  the  acquisition  program,  and  participation  in  the  subsequent 
evaluation  of  how  well  the  resulting  system  satisfies  the  user  com¬ 
mand  requirements.  In  addition,  other  chapters  address  MITRE’s 
work  on  specialty  areas  that  are  important  to  all  effective  C^I 
systems.  These  include  interoperability  with  other  systems,  reliabil¬ 
ity  and  maintainability,  system  availability,  physical  and  electronic 
survivability,  training,  system  performance  monitoring,  support- 
ability,  testing,  and  provision  of  GEE.  Some  of  the  lessons  MITRE 
learned  in  these  areas  are  also  covered. 

MITRE  believes  a  program  was  a  success  if  the  government 
achieved  the  best  capability  possible  for  the  time  and  money  in¬ 
vested.  It  is  always  a  feeling  of  great  pride  and  satisfaction  when 
the  systems  one  has  worked  so  hard  to  perfect  perform  well  in 
helping  to  accomplish  the  intended  operational  mission.  MITRE’s 
systems  engineering  work  provides  ample  opportunity  for  this 
professional  reward.  The  corporate  role  and  the  associated  respon¬ 
sibility  are  the  challenge  and  the  satisfaction  that  the  staff  enjoys. 
They  are  the  reasons  that  dedicated  professionals  work  at 
MITRE — the  challenge  of  the  responsibility  and  the  chance  to  do 
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something  signiticanr  toward  the  achievement  of  important 
national  capabilities. 

.As  systems  engineer,  ,\11TRE  is  prepared  to  take  extraordinarv 
steps  to  help  the  government  achieve  required  capabilities  and 
often  does.  At  the  same  time,  both  the  Ciorporation  and  the 
government  must  observe  the  rules  that  govern  FFRDCis.  .Mll  Rl-. 
cannot  usurp  the  roles  of  either  government  or  industry  and  must 
studiously  avoid  even  the  appearance  of  a  confi.et  of  interest. 
Only  in  these  ways  will  the  special  relationship  between  .MU  RE 
and  the  government  be  preserved  for  future  programs  of  national 
significance. 


Apptidix 


MITRE  Systems  Eimmeering  Documentotioii  and  Tools 

Much  has  been  learned  at  MITRE  over  the  L  st  35  years  about  the 
systems  engineering  of  command,  control,  communications  and 
intelligence  (C^l)  systems.  That  knowledge  resides  in  the  e'  .  erienced 
MITRE  systems  engineering  staff.  It  has  also  been  recorded  in  a  very 
lengthy  list  of  corpo  aie  documents.  Effective  use  by  the  MITRE  staff 
of  the  information  contained  in  that  documentation  can  significantly 
improve  the  Corporation’s  performance  in  the  systems  engineering 
role.  In  addition,  the  existing  documentation  is  a  meaningful  record 
of  the  breadth  and  depth  of  th"  C orporation’s  systems  engineering 
skills  and  of  the  impav,  those  skills  have  had  on  the  success  of  Elec¬ 
tronic  System  Center  (ESC)  acquisition  programs.  For  those  reasons, 
that  documentation  is  worthy  of  some  discussion  here. 

Clearly,  no  brief  summary  can  do  justice  to  the  thousands  of 
MITRE  documents  that  have  been  produced.  On  the  other  hand,  the 
material  cited  here  is  intended  to  help  one  understand  the  scope  of 
MITRE’s  work  in  the  systems  engineering  role.  Beyond  that,  by  citing 
a  series  of  specific  documents,  it  is  hoped  to  reinforce  a  number  of 
key  points  made  in  the  book.  These  include  the  assertion  that  the 
MITRE  staff  is  skilled  in  the  technical  areas  important  to  C’i  systems 


acquisition,  that  the  Corporation  applies  those  skills  to  the  job,  and 
that  MITRE  technical  skills  have  significant  impact  on  the  success  of 
the  acquisition  programs  for  which  the  Corporation  has  the  systems 
engineering  role. 

Much  of  MITRE’s  documentation  is  project-specific.  It  might 
include  a  system  specification,  analysis  of  test  results,  study  of  the 
impact  of  a  proposed  change  in  requirements,  and  many  other  similar 
papers.  However,  there  is  also  a  body  of  technical  documentation 
that  transcends  individual  projects.  The  references  in  this  Appendx 
include  both  project-specific  and  generic  material. 

Existing  MITRE  documentation  may  be  obtained  through 
MITRE’s  Library.  Judicious  use  of  this  information  by  the  technical 
staff  improves  MITRE’s  performance  in  the  systems  engineering  role. 
Library  research  facilities  will  help  to  identify  what  is  available  and 
to  retrieve  it  when  necessary.  Discussions  with  MITRE  staff  experts 
can  facilitate  the  identification  of  the  important  and  current  docu¬ 
ments  in  any  given  area.  A  recent  MITRE  initiative,  called  the  Soft¬ 
ware  Experience  Factory,  is  collecting  MITRE  experiences  in  system 
and  software  acquisition,  storing  them  in  a  central  on-line  database, 
and  making  the  database  accessible  to  the  staff.  This  database  con¬ 
tains  briefings,  letters,  and  memoranda  that  convey  MITRE’s  lessons 
learned  and  information  across  system  acquisition  projects. 

The  Scope  of  MITRE's  Systems  Engineering  Activities 

MITRE’s  activities  as  systems  engineer  for  ESC  C^I  systems  are 
described  in  the  body  of  this  book.  The  approach  used  here  is  general 
in  nature  and  uses  work  on  a  specific  project  only  to  help  illustrate 
particular  points.  However,  a  chronicle  of  MITRE’s  activities  on  a 
specific  major  systems  engineering  project  is  another  way  to  explain 
what  MITRE  does  and  how  that  work  impacts  on  the  C^I  system 
acquisition  program. 

One  of  MITRE’s  major  projects  over  the  last  25  years  has  been  the 
Airborne  Warning  and  Control  System  (AWACS).  The  Corporation 
has  served  as  systems  engineer  on  the  U.S.  AWACS  project  from  the 
beginning  of  the  acquisition  in  the  late  1960s  through  several  major 


upgrades  to  the  system.  MITRE  was  also  the  systems  engineer  on  the 
NATO  and  Saudi  Arabian  AWACS  programs.  In  1990,  the  Corpora¬ 
tion  published  a  document  that  recounts  its  activities  with  the  U.S. 
AWACS  and  the  initiation  of  the  NATO  AWACS  program.'  The 
document  was  written  by  the  MITRE  technical  staff  who  were  the 
key  people  in  the  Corporation’s  work  on  the  programs;  it  covers  the 
basic  rationale  for  AWACS  and  MITRE’s  work  in  preparing  the 
system  specification.  AWACS  had  to  be  sold  both  technically  and 
operationally.  Many  high  level  people  of  varying  opinions  were 
involved  in  program  decisions.  Could  the  radar  be  made  to  work 
satisfactorily?  Was  the  resulting  capability  operationally  useful?  As 
described  in  the  paper,  MITRE  played  key  roles  in  establishing  posi¬ 
tive  answers  to  both  of  these  questions.  The  Corporation’s  many 
contributions  to  the  success  of  the  AWACS  program  are  reviewed  in 
the  referenced  report. 

The  AWACS  project  is  an  excellent  example  of  the  range  of 
mitre  activities  as  systems  engineer  for  a  large  C’l  system.  It  is  also 
a  fine  example  of  the  impact  that  MITRE  can  have  on  such  a  system. 
The  Corporation’s  work  was  crucial  to  the  Department  of  Defense 
(DOD)  and  congressional  decisions  affirming  that  the  system  would 
work  and  that  it  would  be  operationally  useful.  The  validity  of  those 
decisions  has  been  amply  demonstrated  many  times  over  the  years, 
but  no  more  dramatically  than  in  the  Gulf  War. 

Producing  a  report  such  as  the  AWACS  document  described  above 
is  a  time-consuming  and  expensive  process.  For  those  reasons,  not 
many  MITRE  papers  are  specifically  intended  to  describe  the  range  of 
MITRE  activities  in  the  systems  engineering  role.  Another  historical 
example  that  helps  one  to  understand  some  of  the  MITRE  culture  is 
provided  in  a  book  by  j.F.  Jacobs.^ 

Normally,  each  of  the  MITRE  individual  project  efforts  is  docu¬ 
mented  as  it  takes  place.  To  gain  a  project-wide  appreciation  for  the 


'  MITRE  and  AWACS:  A  Systems  Engineering  Perspective,  The  MITRE  Corpora¬ 
tion,  Bedford,  MA,  1990. 

^J.F.  Jacobs,  The  SAGE  Air  Defense  System,  A  Personal  History,  The  MITRE 
Corporation,  Bedford,  MA,  1986. 


Corporation’s  work  requires  a  review  of  many  different  documents. 

There  is,  however,  another  class  of  documents  that  summarizes  MITRE’s 
work  on  major  projects.  Each  year,  the  Corporation  provides  awards  to 
MITRE  personnel  who  have  worked  on  a  government  project  that  has 
reached  some  major  milestone  or  achieved  operational  status.  To  com¬ 
pete  for  these  awards,  a  MITRE  project  leader  must  submit  a  document 
to  management  describing  MITRE’s  work  on  the  project  and  its  contri¬ 
bution  to  the  project’s  success.  These  documents  help  illustrate  the 
Corporation’s  systems  engineering  work  on  a  project-by-project  basis 
and  can  be  obtained  from  the  respective  project  leaders. 

Although  not  limited  to  MITRE’s  work  for  the  Air  Force,  a  DOD 
publication  summarizes  the  very  broad  scope  of  the  Corporation’s 
activities  and  contributions  on  DOD  programs  for  1990  and  1991.^ 

High  Impact  MITRE  Technical  Wark 

The  objective  of  all  of  MITRE’s  work,  of  course,  is  to  have  a  positive 
impact  on  the  success  of  the  acquisition  projects  for  which  the  Corpora¬ 
tion  has  systems  engineering  responsibility.  MITRE’s  systems  engineer¬ 
ing  work  affects  a  system  acquisition  program  in  many  ways.  In  this 
section,  a  few  of  its  most  significant  technical  contributions  are  briefly 
mentioned.  These  examples  help  to  illustrate  the  technical  strength  of 
the  MITRE  staff  and  the  ability  of  the  Corporation  to  apply  that 
strength  to  the  government’s  acquisition  programs. 

MITRE  analysis  and  simulation  studies  resulted  in  a  major  rede¬ 
sign  of  the  AWACS  radome  and  a  significant  improvement  in  system 
performance.**  MITRE  evaluated  the  compromises  between  the  elec¬ 
tromagnetic  propagation  and  aerodynamic  requirements  of  an  early 
Milstar  satellite  communication  aircraft  radome.^  To  aid  in  the  design 
and  development  of  the  installation  of  a  Milstar  terminal  on  an 


^Application  and  Use  of  the  Fiscal  Year  1990  and  Fiscal  Year  1991  Reports  and 
Recommendations  Provided  by  the  MITRE  C’l  FFRDC  to  the  Department  of 
Defense,  Office  of  the  Assistant  Secretary  of  Defense  (C^I),  Washington,  D.C. 

■'  MITRE  and  AWACS:  A  System  Engineering  Perspective,  The  MITRE  Corporation, 
Bedford,  MA,  1990. 

‘  B.F.  Huhin,  Shape  Analysis  of  the  MILSTAR  Raytheon  ABNCP  Radome,  WP 
26744,  The  MITRE  Corporation,  Bedford,  MA,  1986. 


aircraft,  the  Corporation  prepared  a  detailed  description  of  the  modi¬ 
fications  and  interfaces  required  to  the  aircraft  structure,  electrical 
power,  environmental  control,  and  navigation  systems. *’ 

A  critical  performance  requirement  of  the  Joint  Strategic  Target  At¬ 
tack  Radar  System  (Joint  STARS)  radar  is  detection  of  slow-moving 
vehicles.  During  the  early  radar  design  phase,  the  prime  contractor 
proposed  a  simplified  step  scanning  approach  to  beam  scanning  for 
moving  targets.  This  approach  would  have  held  the  antennna  at  a  fixed 
pointing  angle  for  several  radar  bursts  before  moving  the  antenna  a  full 
azimuth  beamwidth.  MITRE  radar  system  engineers  had  previously 
performed  a  trade-off  analysis  indicating  that  a  continuous  scanning 
design,  in  which  the  antenna  is  moved  a  fraction  of  a  beamwidth  on 
every  burst,  would  yield  significant  improvement  in  overall  performance, 
especially  for  low-velocity  targets.  Thus,  the  system  engineer  was  able  to 
quickly  convince  the  contractor  to  accept  the  continuous  scanning  design 
approach.^  The  Corporation’s  research  work  identified  technical  ap¬ 
proaches  that  permit  compression  of  the  synthetic  aperture  radar  infor¬ 
mation  sent  from  the  Joint  STARS  aircraft  to  the  ground.'^  The  approach 
represents  a  reasonable  compromise  between  reducing  the  amount  of 
information  that  must  be  transmitted  and  the  errors  introduced  by  doing 
so.  MITRE  analysis  and  simulation  of  several  candidate  designs  for  the 
improved  Joint  STARS  UHF  radio  system  led  to  a  recommendation  that 
the  contractor  adopted.’  "’ 

In  performing  analyses  of  radar  systems,  MITRE  needs  to  determine 
the  characteristics  of  the  targets  that  the  radars  will  detect.  For  example, 
MITRE  estimated  that  the  radar  cross  section  of  future  targets  for  the 


*  R.  Santos  et  al..  Group  A  Kit  Installation  Package,  WP  29871,  The  MITRE 
Corporation,  Bedford,  MA,  November  1991. 

Radar  Design  Status  (U),  memorandum  6460-B1719,  The  MITRE  Corporation, 
Bedford,  MA,  3  October  i986. 

*  b.W.  Earn,  S.E.  Gordon,  and  J.M  Knight,  Synthetic  Aperture  Radar  Compression 
Using  the  Multi-State  Coding  Scheme,  MTR  10998,  The  MITRE  Corporation, 
Bedford,  MA.  March  1990. 

’J.  Low  and  J.A.  Sasso,  Joint  STARS  Cosite  Design  Technical  Evaluation  Interim 
Status  Report  (U),  WP  29234,  The  MITRE  Corporation,  Bedford,  MA,  January 
1991. 

J.  Low  and  J.A.  Sasso,  Joint  STARS  Cosite  Technology  Evaluation  Study  - 
Interim  Results  (U},  WP  29745,  The  MITRE  Corporation,  Bedford,  MA, 

31  July  1991. 


Advanced  Surveillance  Tracking  Technology  (ASIT)  system.  To  perform 
such  analyses,  the  Corporation  has  had  an  ongoing  set  of  projects  that 
bring  in  new  electromagnetic  calculation  codes  and  enhance  them  to 
allow  practical  and  accurate  calculations  of  the  type  needed.  The  codes 
and  methods  developed  have  been  documented." 

As  the  radar  cross  section  of  airborne  targets  such  as  stealth  air¬ 
craft  and  cruise  missiles  becomes  increasingly  smaller,  larger  power- 
aperture  products  are  needed  to  prevent  clutter  from  interfering  with 
the  target.  As  the  power-aperture  increases,  so  does  the  clutter  level. 
This  is  particularly  true  for  high  pulse-repetition  frequency  radars, 
since  near-in  clutter  competes  with  distant  targets.  Also,  as  the  re¬ 
ceiver  aperture  is  made  larger,  the  susceptibility  to  jamming  increases. 
Receiving  antennas  with  uliralow  sidelobe  patterns  have  been  pro¬ 
posed  for  large  phased-array  radars  to  suppress  interference  signals. 
However,  MITRE  identified  limitations  in  these  techniques  for 
achieving  required  interference  signal  suppressions  and  developed  a 
flexible  end-to-end  simulation  that  can  be  used  to  evaluate  the  capa¬ 
bilities  of  ASTT  adaptive  antenna  radar  designs.'^*'^’''*  The  simulation 
program  enables  systems  engineers  to  perform  complexity-versus- 
performance  tradeoffs  relative  to  the  optimal  space-time  processing 
for  these  implementations. 

As  part  of  its  support  to  various  DOD  over-the-horizon  backscat- 
ter  (OTH-B)  radar  programs  over  the  past  10  years,  MITRE  has 
developed  a  simulation  program  (HFRAD)  that  allows  a  user  to 
predict  the  performance  of  an  OTH-B  radar  under  various  environ¬ 
mental  conditions."  This  program  has  been  calibrated  against  real- 


"  D.P.  Allen  et  al.,  MSR  Project  90880:  Target  Scattering  Year-End  Report,  MTR 
10640,  The  MITRE  Corporation,  Bedford,  MA,  August  1989. 

B.B.  Suresh  and  J.A.  Torres,  Advanced  Airborne  Radar  Simulation  with  Adaptive 
Antenna  Techniques,  Vol.  1,  MTR  92B0000058,  The  MITRE  Corporation, 

Bedford,  MA,  August  1992. 

B.B.  Suresh  and  J.A.  Torres,  Airborne  Radar  Simulation  with  Adaptive  Antenna 
Techniques,  M  92B00000083,  The  MITRE  Corporation,  Bedford,  MA,  July  1992. 

'■*  B.B  Suresh  and  J.A.  Torres,  Investigation  of  Some  Critical  Issues  of  Space-Time 
Processing  (STP)  Affecting  Airborne  Radar  Performance,  MTR  11055,  The  MITRE 
Corporation,  Bedford,  MA,  January  1992. 

''  T.J.  Elkins,  OTH  Radar  Performance  Evaluation,  MTR  10938,  The  MITRE 
Corporation,  Bedford,  MA,  July  1990. 


world  systems;  it  has  enabled  MITRE  to  analyze  the  expected  perfor¬ 
mance  of  OTH-B  radars  that  are  being  designed,  planned  for  at 
possible  sites,  or  tested.  This  capability  has  allowed  the  Corporation 
to  quickly  address  changing  threat  scenarios  and  new  missions.  The 
evolution,  configuration  control,  and  program  verification  have  been 
continually  documented  over  this  period.'*’ 

MITRE  Systems  Engineering  Informntion  ond  Techniques  Stnte-of-the-Art  Surveys 

The  technologies  associated  with  modern  C’l  systems  are  among 
the  most  volatile.  No  field  is  changing  more  rapidly  than  information 
systems — MITRE’s  expertise.  Staying  current  in  any  particular  por¬ 
tion  of  this  technology  is  a  significant  challenge.  In  response  to  that 
challenge,  MITRE  technical  experts  frequently  publish  surveys  of  the 
state  of  the  art  in  their  particular  field.  Some  recent  examples  include 
areas  such  as  electronic  support  measures  (ESM)  technology,'^  data¬ 
base  management  systems,"*  display  systems, graphics  worksta¬ 
tions,-"  and  three-dimensional  graphics  techniques.-'  Since  the 
changes  taking  place  are  both  frequent  and  significant,  there  is  a 
continual  need  to  keep  this  information  up  to  date.  The  state-of-the- 
art  information  provided  helps  staff  members  on  the  various  MITRE 
projects  to  stay  current.  The  documentation  also  serves  to  identify 
MITRE  experts  who  can  be  called  upon  to  help  as  needed. 

MITRE  Tefbnkul  Studies 

In  addition  to  survey  information,  MITRE’s  staff  publishes  papers 
to  report  on  the  technical  results  of  its  work.  Sometimes  this  informa¬ 
tion  is  developed  in  support  of  a  particular  acquisition  project;  at 


T.S.  Lee  et  al.,  HFRAD  Version  508  Documentation,  Vols.  I,  II,  and  III, 

WP  29834,  The  MITRE  Corporation,  Bedford,  .M.\,  1  .March  1992. 

A  General  Survey  of  ESM  Technology,  Vols.  I  and  II,  WP  24849,  The  MITRE 
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'*  D.  Wolfset,  Comparison  of  Database  Management  Systems,  WP  28618,  The 
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Applications,  M89-5,  The  MITRE  Corporation,  Bedford,  MA,  1989. 


other  times,  it  is  developed  as  part  of  MITRE’s  Technology  Program. 
In  any  case,  publication  of  the  results  is  important,  since  so  many 
different  disciplines  are  common  to  a  large  number  of  C’l  systems. 

To  be  efficient,  one  must  apply  knowledge  gained  in  one  program  to 
others  to  which  it  relates.  Again,  a  wide  range  of  subjects  is  covered, 
and  only  a  few  of  them  will  be  cited  here. 

In  an  attempt  to  speed  up  the  process  of  acquiring  needed  capabili¬ 
ties,  an  approach  of  rapid  prototyping  has  been  employed  in  the  last 
few  years.  The  approach  gives  the  user  an  early  look  at  what  may  be 
provided  and  thereby  improves  the  communication  between  user  and 
developer  about  requirements.  It  also  helps  one’s  understanding  of 
how  difficult  providing  the  capability  may  be  and  how  much  it  might 
cost.  MITRE  has  been  applying  this  approach  to  its  early  work  on 
new  systems.  For  example,  on  the  Joint  Tactical  Distribution  System 
(JTIDS),  the  Corporation  has  prototyped  automated  tools  for  the 
generation  of  the  many  system  initialization  parameters  needed  to 
operate  a  network.  This  has  had  three  benefits:  it  proved  that  useful, 
automated  tools  could  be  developed;  it  resulted  in  a  useful  capability 
for  early  testing;  and  it  led  to  a  refined  set  of  requirements  for  the 
acquisition  of  tools  for  operational  use.^^-^^  MITRE  also  assesses  the 
efficacy  of  rapid  prototyping  as  part  of  many  projects  and  documents 
that  experience.^'* 

MITRE  has  been  in  the  forefront  of  the  network  communications 
area  for  over  20  years.  The  Corporation  holds  patents  in  that  area^^ 
and  has  been  instrumental  in  the  application  of  both  local  area  and 
wide  area  networks  to  government  systems.  MITRE  continues  to 
exercise  its  technical  leadership  in  networks  through  the  efforts  of  the 
Corporation’s  Network  Center  personnel.  Some  of  their  publications 

L.E.  Daeke,  JTIDS  Network  Design  Computer  Aid  -  Algorithms,  WP  27770,  The 
MITRE  Corporation,  Bedford,  MA,  July  1988. 
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include  information  on  high  speed  networks’*’  -'  and  on  the  latest 
approach  to  interfacing  diverse  systems.’** 

Another  important  aspect  of  C^I  systems  is  the  display  facilities 
through  which  the  operators  interface  with  the  remainder  of  the 
system.  This  has  always  been  a  technical  skill  area  within  MITRE. 
Recent  publications  by  the  MITRE  display  engineers  include  those  on 
large  screen  display  technology  ’“'and  requirements,  **’  and  liquid 
crystal”  and  virtual  reality  displays. ’’ 

Other  recent  MITRE  technical  publications  of  interest  to  many 
projects  include  those  in  areas  such  as  the  use  of  ESM  data  with 
radar  information,^’  ESM  data  fusion,^"*  and  communications.’’ 

MITRE  "How-to"  Oocufflonts 

In  the  techniques  area,  MITRE  experts  in  various  aspects  of  sys¬ 
tems  engineering  often  document  their  approach  to  a  particular 
discipline  as  a  way  of  helping  other  members  of  the  technical  staff  to 
apply  the  knowledge  and  experience  to  a  new  project.  Although 
providing  step-by-step  “how-to”  explanations  for  individual  systems 
engineering  tasks  is  outside  the  scope  of  this  book,  it  is  important  to 
recognize  that  many  such  MITRE  papers  exist  and  to  illustrate  that 
with  a  few  examples. 
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Computer  security  is  a  complicated  technical  problem  that  per¬ 
vades  most  modern  C  ’l  systems.  It  is  an  area  in  which  the  Corpora¬ 
tion  has  achieved  a  national  reputation  for  excellence.  In  a  MITRE 
paper  on  that  subject,  the  authors  describe  the  most  recent  DOD 
policies  on  information  security.’*  They  go  on  to  explain  the  potential 
system  vulnerabilities  and  alternative  means  for  dealing  with  them, 
and  they  describe  sources  for  guidance  in  implementing  proper  safe¬ 
guards.  Relevant  technology  is  reviewed  and  suggestions  made  for 
how  to  address  information  security  in  each  program  phase.  Required 
testing  is  discussed  in  some  detail.  The  role  that  MlTRE’s  Informa¬ 
tion  Security  Center  is  prepared  to  play  on  a  program,  if  required,  is 
also  reviewed.  The  document  describes  the  problem,  provides  sugges¬ 
tions  for  dealing  with  it,  and  offers  additional  help.  It  is  a  good 
reminder  to  MITRE  project  personnel  of  the  importance  of  computer 
security  and  a  useful  guide  on  how  to  deal  with  it. 

Another  challenging  area  that  pervades  all  C^I  systems  is  soft¬ 
ware.  Software  development  is  labor-intensive  and  difficult;  as  a 
result,  planning  for  software  acquisition  and  reviewing  software 
products  are  the  focus  of  considerable  technical  and  management 
time  and  attention.  MITRE  has  been  very  active  in  these  attempts 
to  improve  the  acquisition  of  software.  The  Corporation  formed  a 
technical  center  for  software  and  its  staff  have  produced  a  number 
of  documents  related  to  how  to  acquire  effective  software.  These 
include  an  overview  of  the  relevant  government  policies,  regula¬ 
tions,  and  standards  for  software  acquisition.^^  The  document  is 
used  in  ar.  in-house  training  program  for  MITRE  staff  members.  A 
related  and  more  detailed  paper  provides  guidance  for  tailoring  the 
DOD  software  development  standard  to  a  particular  system  acqui¬ 
sition.’**  Other  papers  provide  guidance  in  the  preparation  and 
review  of  software  requirements,  design,  and  software  quality 
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contrt)!. Other  software-related  MITRF.  papers  cover  such 
areas  as  object-oriented  programming,^’’ computer-aided  software 
engineering  (CiAbE)  tools,^"*  use  of  software  metrics  in  evaluating 
progress  in  software  development,"*'  and  application  of  the  Soft¬ 
ware  Engineering  Exercise  as  a  tool  in  contractor  selection. "*'' 

The  interface  between  the  people  operating  the  system  and  the 
system  hardware  and  software  is  yet  another  especially  important 
design  consideration.  MITRE  experts  in  that  area  have  developed  an 
approach  to  specifying  that  interface."*'  This  approach  embodies  the 
experience  MITRE  has  gained  with  existing  systems. 

As  previously  noted,  MITRE  plays  an  important  role  in  source 
selection,  a  very  difficult  and  sensitive  activity.  Wher  wins  and  who 
loses  is  obviously  important  both  to  the  companies  involved  and  to 
the  eventual  achievement  of  the  needed  system  capability.  Because  it 
is  so  important,  MITRE’s  role  in  it  has  been  documented"***  and  the 
contents  used  by  the  MITRE  Institute  to  train  MITRE  staff  to  partici¬ 
pate  in  source  selections.  Both  technical  and  ethical  considerations 
are  included. 

MITRE,  of  course,  is  involved  throughout  the  development  cycle  that 
follows  source  selection.  Capturing  the  best  practices  for  system  engi¬ 
neering  is  the  objective  of  other  MITRE  how-to  documents,  such  as  one 
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on  partitioning  computer  software.’**'  There  are  many  other  examples  of 

1 FRE  how-to  documents.  They  range  from  the  strictly  technical  to  the 
more  management-oriented  tasks.  Their  judicious  use  helps  to  improve 
the  efficiency  and  effectiveness  of  MITRE’s  systems  engineering  work. 

MITRE  SystMM  E»ghteriB9  Tools 

A  significant  portion  of  MITRE’s  work  on  an  acquisition  project  can 
be  based  on  the  training  and  experience  of  its  technical  staff.  However, 
as  in  any  profession,  staying  current  with  the  rapidly  changing  technol¬ 
ogy  and  assessing  the  new  technology  for  potential  application  to  C*I 
systems,  requires  that  the  MITRE  staff  perform  technical  analyses,  con¬ 
duct  experiments,  make  measurements,  or  carry  out  simulations.  To  do 
such  work  requires  the  time  and  effort  of  the  staff  members,  as  well  as 
the  facilities  necessary  to  perform  such  tasks.  As  discussed  in  Chapter  4, 
this  need  is,  in  part,  filled  by  the  MITRE  Technology  Program.  However, 
such  work  is  also  often  necessary  as  a  part  of  an  acquisition  project.  This 
work  is  referred  to  as  design  verification,  and  the  rationale  for  it  is  also 
discussed  in  Chapter  4.  A  few  examples  of  the  impact  of  such  work  are 
cited  here. 

MITRE’s  analysis  and  computer  simulation  work  on  AWACS 
mentioned  above  resulted  in  a  redesign  of  the  radome  and  an  impor¬ 
tant  improvement  in  the  system  performance.  The  Corporation’s  Joint 
STARS  radar  evaluation  facility  was  instrumental  in  the  design  of  a 
tracking  logic  for  ground  targets  as  they  moved,  stopped,  or  were 
obscured  by  terrain  features. This  problem  is  very  different  from 
that  of  tracking  aircraft  as  seen  by  a  platform  such  as  AWACS.  The 
use  of  a  simulation  facility  enabled  MITRE  to  define  an  implement- 
able  approach.  The  Corporation’s  analysis  work  on  Joint  STARS 
simulation  models  of  the  data  processing  and  display  subsystems 
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identified  and  predicted  bottlenecks  and  shortfalls  in  processing 
capacity.’’  Identification  of  prtKessing  shortfalls  led  to  major 
upgrades  in  both  the  main  data  processing  computers  and  the 
operator  workstations. 

As  the  e.xamples  discussed  in  this  Appendix  suggest,  .MITRE  loci.- 
mentation  helps  to  illustrate  the  systems  engineering  techniques  that 
have  proven  successful  on  major  C*1  acquisition  projects  for  ESC. 
MITRE  documentation  is  also  a  key  factor  in  making  the  information 
on  the  latest  information  system  technology  and  techniques  available 
to  a  broad  spectrum  of  its  staff.  The  existing  documentation  reflects 
the  Corporation’s  professional  approach  to  its  systems  engineering 
responsibilities. 
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Glossary 


Acronym 

Definition 

AFMC 

Air  Force  Materiel  Command 

AFSC 

Air  Force  Systems  Command 

ALRI 

Airborne  Long  Range  Radar  Input 

AWACS 

Airborne  Warning  and  Control  System 

BITE 

Built-in  Test  Equipment 

BUIC 

Backup  Interceptor  Control 

CM 

Command,  Control, 

Communications,  and  Intelligence 

CDRL 

Contract  Data  Requirements  List 

DOD 

Department  of  Defense 

ESC 

Electronic  Systems  Center 

ESCR 

Electronic  Systems  Center  Regulation 

ESD 

Electronic  Systems  Division 

ESM 

Electronic  Support  Measures 

FAR 

Federal  Acquisition  Regulations 

FCRC 

Federal  Contract  Research  Center 

FFRDC 

Federally-Funded  Research  and 

Development  Center 

GFE 

Government-Furnished  Equipment 

IV&V 

Independent  Verification  and  Validation 

Joint  STARS 

joint  Strategic  Target  Attack  Radar  System 

JTIDS 

Joint  Tactical  Information  Distribution 

System 

MIT 

Massachusetts  Institute  of  Technology 

MSR 

MITRE  Sponsored  Research 

NADGE 

NATO  Air  Defense  Ground  Environment 

NASA 

National  Aeronautics  and  Space 

Administration 

NORAD 

North  American  Aerospace  Defense  Command 

OEP 

Operational  Employment  Plan 

OMB 

Office  of  Management  and  Budget 

PMD 

Program  Management  Directive 

R&D 

Research  and  Development 

RFP 

Request  for  Proposal 

Acronym 

Definition 

SAC 

Strategic  Air  Command 

SACDIN 

Strategic  Air  Command  Digital  Information 
Network 

SAGE 

Semi-Automatic  Ground  Environment 

SETA 

System  Engineering  Technical  Assistance 

SOW 

Statement  of  Work 

SPD 

System  Program  Director 

SPO 

System  Program  Office 

TAG 

Tactical  Air  Command 

TBS 

Training  and  Battle  Simulation 

fEMS 

Technical  Engineering  Management  Services 

TO&P 

Technical  Objectives  and  Plans 

